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Executive Summary 

Reason for Study 

Significant progress has been made in recent years toward the provision of data- 
linked weather information to pilots. Recently, the FAA entered into a government- 
industry partnership to develop the broadcast Flight Information Services Data Link 
(FISDL) service, which is scheduled for an initial operational capability in 2000. 
There are also non-FAA FISDL systems under development, some of which use re- 
quest-reply links. However, many questions remain unanswered about the form and 
function of airborne data link weather information. Some specific issues include: 

• Appropriate use of data-linked weather information for in-flight decisions 
(strategic and/or tactical); 

• Use of data-linked weather information as a supporting in-flight information 
source and not as a sole source of in-flight weather information; 

• Display of real-time ownship aircraft position and/or flight path information 
on cockpit displays of time-delayed weather products. 

This study was conducted as the first in a series of studies to examine such issues and 
to develop a better understanding of the use of data-linked weather information. The 
results of such studies serve to validate the existing FAA guidance, RTCA standards 
and training materials for cockpit weather systems, and to provide recommendations 
for future guidance. 


Overview of Study 

The study reported herein is the first in a series of rigorous investigations using pi- 
loted simulation of the effects of data link weather displays upon pilot decision per- 
formance. An experiment was conducted with twenty-four current instrument rated 
pilots who were divided into two equal groups and presented with a challenging but 
realistic flight scenario involving weather containing significant embedded convec- 
tive activity. The flight scenario was intentionally designed to present significant 
stress for pilot decision-making in a challenging weather environment. All flights 
were flown in a full-mission simulation facility in simulated instrument meteorologi- 
cal conditions. Visibility for the pilot was essentially zero from shortly after takeoff 
until just before landing. The experimental objective was to investigate the potential 
for misuse of weather information on their cockpit displays, and to incorporate the 
lessons learned in recommendations for the design and use of these displays. 

The control group performed the flight with access to conventional sources of pre- 
flight and in-flight weather products. In addition to the conventional weather sources, 
the treatment group was provided with a weather display in the cockpit that presented 
text and graphical weather products. 
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Conclusions of Study 


Test configuration of cockpit weather display did not improve decision-making 

The configuration of the weather display system implemented in this study did not 
improve the decision making of the pilots using the display. Reasons for this finding in- 
cluded: 

1. Pilots were unable to easily perceive their proximity to potentially hazardous con- 
vective weather conditions depicted graphically by the display. 

2. Pilots were unable to easily estimate the juxtaposition of their flight path with the 
path of graphically depicted hazardous convective weather conditions (NEXRAD 
mosaic images) because own path was not depicted on display and probable path 
of the hazardous weather was not depicted. 

3. Use of the cockpit weather display increased the workload for at least half of the 
pilots, decreasing the time available for decision making. 

4. Many pilots experienced difficulty in deciphering METAR textual data. 


Pilots with the cockpit weather information display, relied less on other available 
weather information sources 

The display of NEXRAD mosaic images substantially increased the pilots’ awareness of 
the general location of convective weather in their vicinity. The compelling nature of the 
display of these images, however, caused some pilots to depend too heavily on the 
weather display for their information regarding hazardous convective weather conditions. 
As a result, they failed to obtain other essential and corroborating information from other 
available sources. 


Standards are needed for a cockpit weather information display 

Standards for system performance, display content and format, and operational use do not 
yet exist and must be developed in a timely fashion to support the safe implementation 
and effectiveness of cockpit weather information displays. These standards must be based 
on substantial and well-founded research in lieu of trial and error in the market place. 


Training is required to effectively use a cockpit weather information display 

Substantial training in the use of a cockpit weather display system will be required to 
help pilots understand the limitations of a weather display and its data, to reduce the 
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workload otherwise required to access and interpret weather information, and to enable 
the pilot to fully exploit the potential safety contributions of the display. 


An autopilot may be required to use a cockpit weather information display in 
instrument conditions 

The safe and effective use of a cockpit weather information display in actual instrument 
conditions will almost certainly require the support of an autopilot for most pilots. 


Recommendations 

Recommendations are provided based on the findings of this study for possible incorpo- 
ration in the FAA Aeronautical Information Manual (AIM) and other FAA guidance for 
users and manufacturers of a cockpit weather information display. 

The following recommendations are provided for the consideration by cockpit weather 
information display system manufacturers: 

• Consider Providing Ownship Information 

• Provide Direction and Rate of Hazardous Weather 

• Provide Distance Determination 

• Provide Intuitive NEXRAD Image Age Information 

• Provide METAR Code Translation 

Recommendations are also provided for further research and development efforts, 
including: 

• Further development of weather information integration and display design enhance- 
ments, 

• Further evaluations of specific FISDL issues to support standards for design, imple- 
mentation and use of FISDL services, 

• Development of a training curriculum for weather information displays. 

Recommended information to be added to the AIM and to advisory circulars now in draft 
stage includes: 

• The requirement that pilots become fully proficient in determining and maintaining 
awareness of the age of all FISDL service products, 

• Limitations on the use of FISDL service products due to their age, 

• Further warning that a cockpit weather information display cannot be used for navi- 
gation purposes, 

• The value of an autopilot to offset workload and free up mental processes to support 
use of a cockpit weather information display. 
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1 Introduction 


Statistics indicate that there is, on average, one fatal general aviation accident per day in 
the United States alone. 1 Some of the reasons for these fatalities include pilot-related 
causes, mechanical failure, midair collisions and other problems. 2 3 While mechanical fail- 
ure accounts for only 14.1 percent of the total accidents, pilot-related causes account for 
over 73 percent of the total accidents. The primary causes of fatalities were weather, ma- 
neuvering flight and approaches. Weather-related accidents were more likely to be fatal 
than any of the other major causes of fatal accidents. With an overall fatality rate of 83.1 
percent, weather accidents were the deadliest of the pilot-caused fatalities. Most fatalities 
involving weather were the result of controlled flight into terrain or other objects, spatial 
disorientation leading to uncontrolled flight, or pilot-induced structural failure of the air- 
craft. Some accidents attributed to other causes involved weather as a contributing factor 
as in the cases of improper IFR approach accidents. Windshear and crosswind also 
caused weather-related accidents. Most troubling is that 72.2 percent of the weather- 
related fatalities were caused by attempted VFR flight into Instrument Meteorological 
Conditions (IMC). 

While pilot training and certification regulations to minimize pilot error have been im- 
plemented, there have been significant advances in technology that can offer advanced 
weather displays in the cockpit via data link. This could amount to a significant advance 
in aviation safety. Conventional round dial instruments accompanied by aeronautical 
charts, approach charts, and flight service station briefs represent a few of the many sepa- 
rate pieces of data that must be accessed for safe flight. The pilot is obliged to integrate 
these various pieces of information into a single mental model of the outside world. This 
represents a very appreciable cognitive workload, and, inevitably, mistakes are some- 
times made. 

Advances in display system design are attempting to reduce the pilot's cognitive work- 
load, by doing much of the integration work for them. These designs are moving toward 
flat-panel displays with terrain, traffic, routing, and weather all overlaid on a single 
screen, thereby fostering a more intuitive mental model of “the big picture” for the pilot. 
By reducing the workload involved in mentally integrating multiple elements, pilots' can 


1 341 fatal accidents occurred in general aviation in 1998 out of a total of 1,679 accidents. The 341 fatal 
accidents resulted in 619 fatalities. Source - 1999 Nall Report, Aircraft Owners and Pilots Association 
(AOPA) Air Safety Foundation. 

2 A full 70 percent of all accidents resulted in little or no injury. However, 73.4 percent of all accidents 
(1,233 total/247 fatal) were attributed to pilot-related causes, 14.1 percent (236 total/1 9 fatal) were attrib- 
uted to mechanical/maintenance problems, with the balance of accidents attributed to midair collisions (14 
total/1 1 fatal), alcohol and drugs (2 total/1 fatal), fuel mismanagement (136 total/9 fatal), off airport injuries 
(i.e. bystander injuries/fatalities from debris and impact) (7 total/5 fatal), pilot incapacitation (3 total/2 fa- 
tal), and homebuilt aircraft (199 total/54 fatal). 

3 Weather-related accidents had an 83.1 percent fatality rate (65 total, 54 fatal), maneuvering flight acci- 
dents had a 46.7 percent fatality rate (135 total/63 fatal) and approach accidents had a 36.5 percent fatality 
rate (96 total/35 fatal). 
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allocate attention elsewhere, particularly to higher level situation assessment, judgment 
and decision making tasks. Extra attention to these tasks should reduce the opportunity 
for error and enable the individual to make more considered decisions. 

However, because human performance research has lagged well behind the display 
manufacturers, many of the performance issues have yet to be determined, and the best 
way to display weather information is not yet clear. Nevertheless, weather information 
(because of its great importance in flight safety) is a prime candidate for early imple- 
mentation in the cockpit. 

The Federal Aviation Administration (FAA) awarded contracts in 1999 for the develop- 
ment of two Flight Information Services Data Link (FISDL) systems. The FISDL sys- 
tems will broadcast text and graphical products via data link for reception and display in 
equipped aircraft. An overview of the FISDL systems is provided in Appendix A. At pre- 
sent, however, the initial FAA guidance is limited to a description of the FISDL system 
included in the AIM and draft material on FISDL display standards being developed by 
the RTCA. The guidance provided in the AIM is reproduced in Appendix R. The experi- 
ment reported here was undertaken to develop a better understanding of the use of a 
cockpit weather information display. 

The Research Triangle Institute's Center for Aerospace Technology, sponsored by the 
FAA and NASA, investigated pilot performance using a new airborne weather display in 
a full mission simulator expressly developed for the study of new cockpit technologies in 
general aviation. An experiment was conducted with instrument rated pilots who were 
presented with a challenging but realistic flight scenario involving weather with signifi- 
cant embedded convective activity. The experiment objective was to investigate the po- 
tential for misuse of weather information, and thus provide guidance to the FAA. A cor- 
responding hypothesis was developed and investigated through experimentation: “De- 
layed weather information data linked to the cockpit display may lead to navigation deci- 
sion errors.” 


1.1 Potential Issues with Datalinked Weather Displays 

One potentially significant, indeed critical issue, in the use of displayed weather is that 
weather products are not displayed in real time as are most other cockpit data including 
the data provided by on-board weather radar. 4 This presents the pilot with complex issues 
of interpretation and prediction. It is not clear, for example, whether pilots will try to ex- 
trapolate, from delayed data, the current position of storm cells, and attempt to weave 
between areas of perceived danger (tactical use), or adopt a more conservative approach 
of longer-term route planning to avoid potential hazards altogether (strategic use). The 
term “perceived” is crucial here, as studies to date suggest that a “keep out of the red” 


4 The latest graphical NEXRAD products will be broadcast to aircraft within one minute of reception from 
the weather service provider, but will be five or six minutes old when received from the weather service 
provider for transmission to the aircraft. 
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heuristic procedure may be adopted when, for example, viewing a NEXRAD baseline 
reflectivity product indicating amounts of rain fall according to a color coding scheme. 

Of course, the cessation of red cells (indicating areas of heavy rainfall) does not imply the 
cessation of peril. Areas of low visibility, turbulence and windshear may not appear as 
coded zones in certain weather products so any such heuristic procedure is a dangerous 
one. 

It has been anticipated by some that pilots might try to use weather information “tacti- 
cally” as though it were “real time” and definitive, instead of delayed and probabilistic, 
possibly getting themselves into trouble. This might lead to pilots that become overconfi- 
dent in their ability to judge exactly where it is safe or unsafe to fly. It is believed that 
“strategic” use of the weather display (using the information to plan a route around possi- 
ble danger zones) would be safer and more appropriate. 

A related issue concerns the explicit provision of predicted weather (e.g., storm cell con- 
figuration, location and movement), such that the mental workload of extrapolation is not 
added to pilots’ tasks. Manufacturers and regulatory agencies may be hesitant in provid- 
ing mathematical predictions and extrapolations of weather data to pilots because there 
may be non- trivial liability issues involved. 

1.2 Survey of Relevant Literature 

The literature pertaining to display of weather information provided by data link to the 
cockpit as it exists today is still in its infancy. The next generation of research must begin 
in order to catch up to rapidly emerging technology. The results of a search of the litera- 
ture relevant to the display of data link weather information are provided in Appendix B, 
Literature Search Results; a summary is provided below. 

Most studies have focused on situational awareness (Hansman, & Wanke, 1989; and Lee, 
1990), and expert/novice strategic decision making, (mostly making go/no go decisions), 
(Driskill, Weissmuller, Quebe, Hand, Dittmar, Metrica, & Hunter, 1997; Dershowitz, 
Lind, Chandra, & Bussolari, 1996; Fisher, Brown, Wunschel, & Stickle, 1989; Wiggins, 
Connan, & Morris, 1995; and Wiggins & O'Hare, 1995). Little has been done to examine 
the possible “tactical” decisions made during flight, and none have looked at this issue in 
a full mission simulator. 

One of these issues is the impact of textual versus graphical presentation of weather in- 
formation on pilot decision making. A particularly relevant study was a comparison of 
textual presentation versus graphical presentation of weather information undertaken at 
the Lincoln Laboratory of MIT (Lind, et. al., 1994) that provided a valuable first step by 
looking at the influence of data-link provided graphical weather on pilot decision making. 
When compared to strictly text information, the graphical information caused pilots to 
become more confident in their assessment of the weather, and to make better Go/No Go 
decisions as well as flight path change decisions. Although very valuable, this study was 
performed in an office setting without a true flight simulator and, therefore, without fac- 
tors that come into play in an operational setting. Decisions were made based on static 
images presented at selected certain points during a scripted scenario. 
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Spatial displays have also been found to improve accuracy over text in presenting infor- 
mation for an analog operation/tactical decision task (Wickens & Scott, 1983). 

All these findings are consistent with the multiple resource theory of attention, and the 
proximity/compatibility principle, which suggest that if an individual is to perform a vis- 
ual-spatial task (such as navigating an aircraft through the airspace), then the information 
needed to perform that task should be presented in a visual-spatial way (e.g. as graphics, 
rather than, for example, a visual-verbal way such as in teletyped weather products). 

Studies conducted to date represent a mere fraction of the studies that the introduction of 
new technologies will need, if progress is to be made to resolve the appreciable number 
of issues that arise. 


2 Participants 

This study was a cooperative effort between the Federal Aviation Administration (FAA), 
National Aeronautics and Space Administration (NASA) and the Research Triangle In- 
stitute (RTI). 

2.1 FAA Data Link Office 

The FAA Data Link Office AND-520 is the prime sponsor for this study. This effort was 
undertaken to support the periodic revisions to the Aeronautical Information Manual 
(AIM) performed by AFS-410 of the FAA, and development of other FAA guidance for 
the manufacture and use of cockpit weather information displays supported by data link 
communications. 

2.2 NASA AWIN Project 

The NASA AWIN (Aviation Weather INformation) project provided additional funding, 
technical support, and contract management for this study in support of the FAA. The 
AWIN project is an element under the Aviation Safety Program Office of NASA Langley 
Research Center, Hampton, Va. 

2.3 Research Triangle Institute 

The study was performed by the Flight Systems Engineering Program of RTI located in 
the Institute’s Hampton, Virginia, office. An RTI consultant from Rensselaer Polytechnic 
Institute assisted in the study design and analysis. Another RTI consultant provided air 
traffic control expertise in the design and execution of the study. 
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3 Methodology 

It was decided to base this study on an experiment in which subject pilots representative 
of probable users of the FISDL products would use the FISDL equipment in typical op- 
erational conditions. The objective of the experiment was to investigate the potential for 
misuse of the weather information, and thus provide recommendations to the FAA. In 
particular, the concern was that delayed weather information datalinked to the cockpit 
display might lead to tactical navigation decision errors. Thus, the experiment was de- 
signed to prove or refute such a hypothesis, while extracting other information useful for 
regulatory and procedural guidance. 

3.1 Experimental Design 

The experiment was designed to have certain desirable properties. It was moderate in 
length (approximately one hour depending on pilot actions) in order to eliminate fatigue- 
related effects. It was made up of sufficiently independent phases to test responses to dis- 
crete weather conditions. The “incident density” was to be plausible and would be de- 
signed to occur while crossing informational boundaries (where most decision related 
errors are more likely to occur). The mission scenario and cockpit simulator were to be 
sufficiently realistic such that the subject pilot would be immersed in the experiment. The 
simulator instrument panel in the control group did not have a weather display. The in- 
strument panel used by the treatment group provided a weather information display. 

The experiment employed a “between-subjects” design, whereby two groups of similar 
subject pilots were divided into control and treatment groups. Performance differences 
between the two groups could then be attributed to differences between the control and 
treatment conditions. This is in contrast to a “within-subjects” design, whereby an identi- 
cal group of subject pilots perform two “equivalent” flights to test the hypothesis. With 
the “between-subjects” approach, it is not necessary to design two functionally equivalent 
flight scenarios, which must be sufficiently different in order to minimize training effects. 
However, twice as many subjects are required and care must be taken to assure equiva- 
lence of the two subject groups using the “between-subjects” approach. 

In addition to determining the impact on navigation decision errors due to delayed 
weather information datalinked to the cockpit display, several co-variables were identi- 
fied, and addressed in the course of the experiment to provide advisory information for 
the FAA and feedback to the avionics manufacturers. For example, it was noted how 
much instruction and practice was required to familiarize and operate the display. The 
pilots were questioned about their understanding of the data content, refresh rate, stale- 
ness, and NEXRAD cell size resolution. Post flight questionnaires were administered to 
determine if the pilots were able to determine both their location and their proximity to 
displayed weather (note: ownship position was not provided on the display used during 
the experiment). Pilots in the treatment group were invited to comment on the utility and 
shortcomings of the display. 
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A simulated flight was designed (using actual recorded weather) that originated in New- 
port News, Virginia, and consisted of two decision points: 1) approaching the Richmond, 
Virginia, airport, and 2) enroute to Wallops Island, Virginia. The actual weather consisted 
of two converging frontal boundaries. One frontal system included convective activity 
moving rapidly west to east across the vicinity of the Richmond airport. Another sys- 
tem — a low-pressure trough — contained convective activity developing along a 
north/south line over the Chesapeake Bay. The first decision emphasized the time-delay 
(temporal) aspects of the weather while the second decision emphasized the spatial as- 
pects. Details of each scenario providing the two decision conditions are given in para- 
graph 4.2. 

3.2 Pilot Selection Process 

Because of the nature of the flight scenario, it was decided to limit subject pilot candi- 
dates to those who were currently instrument rated. To maximize the likelihood of navi- 
gation decision errors (i.e., tactical use of the display instead of strategic use) while per- 
forming the scenario’s mission, the pilots were further selected according to their risk 
aversion tendencies and their knowledge of weather. Risk aversion was measured using a 
PC-based test described in Appendix C, Risk Aversion Test. The risk aversion test is a 
domain independent measure of what a subject pilot does in response to a risk-reward 
opportunity, not of what they say they will do. 

Weather knowledge was measured with a written test, included in Appendix D, Weather 
Knowledge Questionnaire and Key. The test was promoted as a general aviation ques- 
tionnaire to disguise the true nature of the experiment, thereby reducing any tendency for 
a subject pilot to study weather interpretation before the actual simulator trials began. 
There was a total score of 39 possible correct answers. 

Following administration of the risk aversion and weather knowledge pre-screening tests, 
the scores were reviewed to assure sufficient diversity. The mean and standard deviation 
of the raw risk and weather knowledge scores were independently computed, normalized, 
and added for each subject. The highest composite positive scores represent high weather 
knowledge/low risk candidates; the lowest negative scores represent low weather knowl- 
edge/high risk candidates. 

The tests were administered to 57 current IFR-rated pilot candidates. To provide a rea- 
sonable probability that differences in the responses between the control and treatment 
groups would be statistically significant (while working within a very limited budget), the 
experiment was conducted with 24 subject pilots: 12 control pilots ( without the data- 
linked cockpit weather display) and 12 treatment pilots (with the cockpit weather dis- 
play). 
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Figure 3.2-1 depicts the sorted raw risk aversion test results for the 57 candidate subject 
pilots. The test scores are tabulated in Appendix E, Subject Pilots Screening Test Results 
and Statistics. Scores of 3.0 or lower correspond to risk-averse characteristics, that is, pi- 
lots who display a low tendency to take risk. Alternatively, scores of 5.0 or greater corre- 
spond to very high risk individuals. 



Figure 3.2-1. Sorted Risk Aversion Scores 

A histogram of the risk aversion test scores is provided in Figure 3.2-2. The scores have 
been quantized to multiples of 0.4. The mean risk score was 3.79, with a standard devia- 
tion of 0.75. 



Figure 3.2-2. Risk Histogram 
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Figure 3.2-3 depicts the sorted weather knowledge scores for the 57 candidate subject 
pilots (see Appendix E, Subject Pilots Screening Test Results and Statistics). Of the 39 
weather-related questions, three candidate subject pilots scored a near perfect score of 38. 
The lowest weather knowledge test score was 12 correct answers. 



A histogram of weather knowledge test scores is provided in Figure 3.2-4. The scores 
have been quantized to multiples of 5.0. The average weather knowledge score was 27.2, 
with a standard deviation of 6.2. 


Weather Knowledge Test Scores Histogram 

(scores quantized to multiples of 5.0) 



Figure 3.2-4. Weather Knowledge Test Scores Histogram 
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The mean and standard deviation of the raw risk scores and weather knowledge scores 
were independently computed, normalized, and added for each subject. The results are 
depicted in Figure 3.2-5 and summarized in Appendix E, Subject Pilots Screening Test 
Results and Statistics. The 24 subjects having the 12 highest and 12 lowest scores are 
highlighted. 



Subject Number 


Figure 3.2-5. Sorted z-Score Sums (Risk + Weather Knowledge) 

A histogram of the combined risk aversion/weather knowledge z-score sums is depicted 
in Figure 3.2-6. The scores have been quantized to multiples of 0.5 (a half standard de- 
viation). 


(Risk + Wx Knowledge) Histogram 

(Scores quantized to multiples of 0.5) 



Figure 3.2-6. (Risk + Weather Knowledge) Histogram 
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A stratified random selection was performed to populate the control and treatment 
groups. The individual with the highest combined, normalized score went into treatment 
group #1. The individual with the second highest score went into control group #1. The 
third highest went into treatment group #1, the fourth highest into control group #1, etc. 
Similarly, the individual with the lowest combined score went into treatment group #2, 
the second lowest into control group #2, and so forth until the four groups were populated 
with six pilots apiece. In addition, two alternate subjects were identified for each of the 
four groups (similarly selected) to accommodate any contingency “no-shows” from 
among the primary candidates (in fact, after having been selected and scheduled, two of 
the primary 24 pilots were not able to participate in the experiment, and were replaced by 
their alternates). 

Thus, following the pilot selection process, a treatment group of 12 pilots (subdivided 
into two groups based on their combined risk/weather knowledge scores), and a control 
group of 12 pilots (similarly subdivided), were selected to participate in the experiment. 
The subject pilot distribution is illustrated in Figure 3.2-7. After the experiment, an 
analysis was performed to determine if the subject pilot’s risk/weather knowledge had 
any influence on tactical navigation decision errors. 

During the conduct of the experiment, an additional subject pilot was invited to partici- 
pate in case it was decided to discard the results obtained from one particularly anoma- 
lous pilot who did not completely “buy-in” to the mission scenario. Thus, a total of 
twenty-five mission simulations were conducted. 


Control Group 

(No Cockpit Weather Display) 

Cl (6 subjects) 

Low Risk 

High WeatherKnowledge 


C2 (6 subjects) 

High Risk 

Low WeatherKnowledge 

Treatment Group 
(With Cockpit Weather Display) 

T1 (6 subjects) 

Low Risk 

High Weather Knowledge 


T2 (6 subjects) 

High Risk 

Low Weather Knowledge 


Figure 3.2-7. Subject Pilot Distribution 
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3.3 Experimental System 

The experiment was performed in a full-mission flight simulation in order to provide a 
realistic operational environment. Two major components comprise the experimental 
system: pre-flight planning tools and flight simulation facility. 

3.3.1 Pre-Flight Planning Tools 

Each pilot was given 30 minutes to plan the flight. The following flight planning tools 
were provided: 

• A written transcript of a telephone Flight Service Station (FSS) weather 
briefing (included in Appendix F, Preflight Weather Briefing) 

• Aircraft Flight Manual 

• Aeronautical charts (sectional and low-altitude enroute) 

• Blank flight logs 

• Partially completed flight plan forms (each pilot given same route). 

3.3.2 Flight Simulation Facility 

The flight simulation facility consisted of a full-mission simulator that provided a simu- 
lation of a complex, high-performance single-engine, single-pilot IFR-equipped airplane 
having the major features and performance of a Piper Malibu PA-46-3 1 OP. The key ele- 
ments of the simulation facility used are illustrated in Figure 3.3.2- 1 and Figure 3. 3. 2-2. 
This full-mission simulator facility consisted of three major sections as follows: 

• Aircraft Cockpit Simulator - Consisted of the cockpit mockup with controls, instru- 
ments, radios and indicators. A closed-circuit television camera was mounted behind 
and above the pilot’s left shoulder to provide live images from the cockpit to the Sce- 
nario Controller and Observer positions. The simulated cockpit instrumentation is 
shown in Figure 3. 3. 2-3. For the treatment group, the weather display was located 
between the primary and secondary instruments to maximize its visibility and prob- 
ability of use. The display was removed for the control group. 

• Simulation Facility and Scenario Controller and Observer Positions - Consisted of 
the master control station used for scenario generation and for selection, monitoring 
and recording of flight progress. It provided the operator with displays of all control 
positions, radio and instrument switch positions, instrument displays and the Out-the- 
Window scene (as presented to the subject pilot). A weather data display consisting of 
NEXRAD images was provided for the scenario controller, and enabled the observer 
to track the video flight’s progress relative to the weather. A video image of the cock- 
pit from the camera was provided to enable the observer to monitor the subject pilot’s 
actions. Live audio of all radio transmissions between the pilot and the Air Traffic 
Controller, Flight Watch, ATIS, etc., was available to the simulation scenario con- 
troller and to the observer. An intercom audio network was provided which permitted 
private conversations between the scenario controller, observers, and air traffic con- 
troller positions. The ability for the pilot and air traffic controller to communicate was 
also provided by the same intercom system. All intercom traffic was recorded on the 
audio track of the video recording. 
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• ATC Controller Position - Consisted of a custom ATC workstation developed for 
performing experiments of this type and a weather display that provided the latest 
NEXRAD images enabling the ATC Controller to track the flight’s progress relative 
to the weather. A display of the current pilot-selected communication frequencies was 
also provided so that the ATC controller could verify that the pilot was contacting 
ATC on the correct frequency before responding to an initial contact. 

Additional detail regarding the experimental system is provided in Appendix G, Cockpit 
Research Facility Description. 



Figure 3.3.2-1. Experimental System Facility Layout 
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Figure 3.3.2-2. Key Simulation Facility Stations 
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Figure 3.3.2-3. Cockpit Simulator Instrumentation 


3.3.3 Computer Recorded Data Items 

All essential data items were collected at a rate of 7.5 hertz by the scenario control com- 
puter during the conduct of the mission scenario. A list of the data items collected is pro- 
vided in Appendix G, Cockpit Research Facility Description. 

3.3.4 Weather Information Display 

The major features of the weather display used in the experiment are depicted in figures 
3.3.4- 1 and 3. 3. 4-2. Details of the FISDL system of which this display is a part are pro- 
vided in Appendix A, Flight Information Services - Broadcast Description. 
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Figure 3.3.4-1. Weather Information Display Controls 
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Figure 3.3.4-2. Weather Information Display Screen Labels 
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4 Procedure 


4.1 Key Phases of Experiment Procedure 

The experiment procedure consisted of the following five phases: 

- Mission briefing 

- Simulator familiarization 

- Pre-flight planning 

- Simulator mission 

- Post-mission briefing 

4.1.1 Mission Briefing 

The subject pilots were given a briefing of the mission objective, mission scenario, and 
an overview of the simulator. The pilots that would be using the display were also given 
an overview of the display operation. The briefing scripts are included in Appendix H, 
Subject Pilot Pre-flight Briefings and Simulator Training. 

4.1.2 Simulator Familiarization 

All subject pilots were provided with a familiarization session and practice flight in the 
simulator. Systems, controls, and displays were explained and demonstrated. The trainer 
answered any questions that the subject pilot had with respect to the operation of the 
simulator. The pilots that would be using the display (groups T1 and T2) were given a 
hands-on training session on the use of the in-flight weather display system. The training 
provided an interactive environment that gave a thorough understanding of the equipment 
and its capabilities. To assure equal treatment to all subject pilots, the training session 
was heavily scripted and the pilots were trained to a predetermined performance level 
derived from the FAA Practical Test Standards for Instrument Pilots. 

4.1.3 Pre-Flight Planning 

Each pilot was given 30 minutes to plan the mission. Weather reports and flight planning 
materials were provided. Additionally, a partially completed flight plan form was pro- 
vided that had the route and aircraft specific particulars completed. 

4.1.4 Simulator Mission 

The pilots were left alone in the simulator for the mission and observed remotely. The 
mission lasted approximately one hour, depending on the pilot and route selected around 
the hazardous weather conditions. Three qualified observers participated in the conduct 
of the experiment. Their responsibility was to observe and provide a record of the pilot’s 
performance in the execution of the experiment mission. The Observer form is included 
in Appendix I. 

4.1.5 Post-mission De-briefing 

Upon completion of the mission, each pilot was given an Immediate Reaction Question- 
naire, included in Appendix J, while still seated in the simulator. After completing the 
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questionnaire, the pilot was interviewed by the experiment observers using the Structured 
Interview Guide found in Appendix K to confirm behavioral actions and decisions. As a 
last requirement, the pilot was given an open-ended questionnaire. Subject pilots in the 
treatment group received the Weather Display Questionnaire included in Appendix L. 
Subject pilots in the control group received the AWIN Study Questionnaire included in 
Appendix M. 

4.2 Mission Scenario 

The mission scenario consisted of a flight to deliver medication to a diabetic patient at the 
NAS A- Wallops facility. The NAS A- Wallops facility is located on the eastern shore of 
Virginia. The insulin available in the Wallops area was tainted and a new supply was to 
be taken to the patient. A possible complication from the diabetes is diabetic ketoacidosis 
(DKA), a common and potentially fatal complication. Mortality from DKA runs from 5 
to 15 percent in various studies. If ketoacidosis develops, one effective therapy is a spe- 
cial form of sodium bicarbonate. Thus, the medical rationale involved the delivery of a 
vital medication, insulin, and a desirable medication, sodium bicarbonate. The pilot was 
informed that a medical mercy mission had been coordinated and that he/she was to be 
the pilot. 

The flight originated at the Newport News (Virginia) airport, with the insulin already on- 
board the aircraft. The pilot was instructed to fly to Richmond, (Virginia) and pick-up the 
special sodium bicarbonate on the way to Wallops Island. 

The pilots were told that their supervisor for operations has already cleared the flight to 
the NASA facility and that the insulin had already been loaded into the aircraft. 

In the course of the preflight briefing, the pilot found that there was a weather front 
moving into Richmond, but that the forecast for the area would permit the pilot to land at 
the Richmond airport to pick-up the sodium bicarbonate medicine. The forecast weather 
for the entire flight placed the aircraft in instrument meteorological conditions, but the 
weather at Wallops Island airport was forecast to be above minimums. 

All flights were flown in a full-mission simulation facility in simulated instrument mete- 
orological conditions. Visibility for the pilot was essentially zero from shortly after take- 
off until just before landing. The pilots were to conduct the flight in accordance with all 
appropriate ATC procedures in conjunction with an Air Traffic Controller (ATC) (lo- 
cated in an adjoining room). The ATC workstation fulfilled the roles of clearance con- 
troller, ground controller, tower controller, approach/departure controller and FSS briefer 
as required throughout all phases of the flight. The experiment observers, air traffic con- 
troller, simulation supervisor, etc. were aware of whether a subject pilot was in the con- 
trol or treatment group by virtue of the presence or lack of the cockpit weather display. 
Otherwise, they were unaware of the risk/knowledge status of the subject until after com- 
pletion of the experiment. The Air Traffic Control Scripts are included in Appendix N. 
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The pilot was able to access the normal in-flight weather services through the simulator 
radios, including: 

• FSS - Flight Service Station 

• FW - Flight Watch 

• ATIS - Automatic Terminal Information Service 

• ASOS - Automated Surface Observation System 

• AWOS - Automated Weather Observation System 

The ATC workstation presented the Air Traffic Controller with a readout of the fre- 
quency that the subject pilot selected on the simulator communication radio. When the 
subject pilot tuned the radio to a frequency that corresponded to a recorded weather mes- 
sage (ATIS, etc.), the scripted weather message was read to the pilot by the Air Traffic 
Controller (role playing) and repeated as long as the subject pilot stayed tuned to that fre- 
quency. If the pilot called either a Flight Service Station or Flight Watch briefer, the Air 
Traffic Controller again read a scripted briefing to the pilot. These weather scripts can be 
found in Appendix O, Enroute Weather Information Scripts. 

In addition to the previously mentioned services, pilots in the treatment group had the 
weather information display with which to obtain updated weather text and graphics. 

Actual weather data was used to assure the realism of the operational scenario. All 
weather information used in this experiment was recorded from actual weather conditions 
that existed in the geographical area of the experiment on the evening of April 25, 2000. 
The NEXRAD images were recorded during passage of multiple weather fronts through 
southeastern Virginia from a prototype FISDL system provided to Research Triangle In- 
stitute by Honeywell, Inc. The NEXRAD images were replayed on the weather display in 
the simulation facility cockpit. All NEXRAD mosaic images used in the experiment were 
recorded with a cell resolution of 4 kilometers. To realistically reproduce actual in-flight 
weather products, the subject pilot received the NEXRAD Radar mosaic images delayed 
by seven minutes. The pilot’s weather display of NEXRAD images were initially seven 
minutes old, aging to 14 minutes old before receipt of the next update (of a seven minute- 
old image). The pilot also had access to graphical and textual Aviation Routine Weather 
Report (METAR) information. 

The NEXRAD weather display used by the Air Traffic Controller was meant to simulate 
(in spirit) the ASR-9 weather radar. The Air Traffic controller would receive a real-time 
NEXRAD image that would then age for seven minutes prior to a new real-time update 
of the NEXRAD image. 

All other weather data products needed to support preflight and in-flight weather reports 
for the experiment scenario were collected from the appropriate FAA sources for the 
same location, date, and time captured in the NEXRAD mosaic images. 
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4.2.1 First Leg of Flight - Newport News to Richmond 

During the course of the first leg of the flight, between Newport News and Richmond, the 
ceiling and visibility at the Richmond airport had descended to below minimums (200 
feet) sooner than forecasted. Additionally, there was a thunderstorm approaching the 
Richmond airport. The only way the pilot could learn of these deteriorating conditions 
would be to obtain in-flight weather updates. Both pilot groups could use the radio to 
gather updated weather, but the experimental group also had the ability to obtain 
NEXRAD and METAR weather information updated through the weather display. 

Before reaching the initial approach fix for the Richmond airport, the weather display de- 
picted a thunderstorm cell several miles to the west of the airport but headed toward the 
airport. See Figure 4.2. 1-1. This image on the pilot’s weather information display was a 
minimum of seven minutes old and could have aged up to 14 minutes old. By the time the 
pilot began the approach, the actual weather cell would have intensified and moved closer 
to the airport. [The ATC workstation weather display showed the storm a couple miles 
northwest of the airport.] There were several possible responses to this scenario. The pilot 
could continue the approach with old data and proceed right into the thunderstorm (poor 
decision), or, the pilot could decide to abandon the approach into Richmond and proceed 
directly to Wallops (good decision). A third option had the pilot asking ATC to provide a 
hold until available information could be obtained and sorted out before deciding to con- 
tinue into the Richmond airport or proceed to Wallops (good or poor decision depending 
on proximity of flight path to thunderstorm). 

As the aircraft traversed the various precipitation zones - as depicted in the level of pre- 
cipitation returns on the simulator operator’s NEXRAD display (which was identical to 
the ATC display) - the simulator operator introduced levels of turbulence appropriate to 
the conditions. For flight in clear air, turbulence was not encountered, but when the air- 
craft traversed into an area depicting precipitation, a turbulence model was applied to the 
simulation and the turbulence was increased in proportion to the depicted hazard. 

The turbulence model is described in Appendix G, Cockpit Research Facility. 

If the pilot gathered weather information (either via voice or from the weather display) 
during the leg between Newport News and Richmond, the pilot was apprised of the rap- 
idly changing weather and had to make a decision to either divert to Wallops, or continue 
the approach into Richmond. This is the judgement call that the experiment was designed 
to uncover along with the basis for the decision by the subject pilot. 

If the pilot proceeded with the approach into Richmond, typical and consistent weather 
warnings were given to the pilot from ATC, including a windshear warning when the pi- 
lot contacted the tower. To expedite the simulator mission, if the pilot decided to proceed 
with a landing into Richmond, ATC informed the pilot (when crossing the final approach 
fix) that the Richmond airport manager had closed the airport due to windshear and heavy 
lightning activity. This methodology would preserve the timing aspects between the air- 
craft position and weather movement as applied to all the test subject flights. Therefore, 
all the pilots either broke-off the attempt to land at Richmond at various distances from 
Richmond, or were waved-off at the final approach fix. 
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This part of the experiment was designed primarily to determine the pilot’s judgment 
relative to the time-related (temporal) issues in the use of the weather information dis- 
play. 



Figure 4.2.1 -1. Display Image When Approaching Richmond 


4.2.2 Second Leg of Flight - Richmond to Wallops Island 

During the leg between Richmond and Wallops, a line of storm cells materialized across 
the direct route to Wallops, with one storm cell to the north of the direct course and one 
to the south. The location of this convective activity can be seen in Figure 4.2. 1-1. The 
distance between the red cells was approximately 10-12 miles. The hole between the 
storms was tempting enough to create a corridor between the areas of hazardous weather. 
These cells did not move substantially with succeeding NEXRAD images, but slightly 
changed shape and size. The METAR graphical and textual depiction showed that the 
Wallops airport was above minimums, therefore giving the pilot an incentive to proceed 
with the flight to Wallops. 

The pilot was monitored as to the decision to proceed between the storm cells, or circum- 
vent the area of thunderstorm altogether. This part of the experiment was designed pri- 
marily to determine the pilot’s judgment relating to spatial interpretation issues in the use 
of the weather information display. 
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5 Results 


5.1 Participants 

A total of 25 pilots participated in this study. Of these 25 pilots, 13 flew with a prototype 
weather display and 12 flew without the weather display to enable the experimenters to 
discriminate between decisions made with and without this potential aid. Participants 
were pilots qualified for and current in instrument conditions with varying levels of expe- 
rience. The mean number of flight hours was 1623.2, with a standard deviation of 1445.2 
(n=25). 

5.2 A Representative Flight from the Data Set 

The ground track of each session was recorded and plotted. The path of subject #7 is pro- 
vided in Figure 5.2-1 as an example of the data collected for each session. Subject #7 was 
equipped with the display. As can be seen in the figure and from examination of the other 
data available in this report, subject # 7 decided not to attempt to land at Richmond be- 
cause of his understanding of the deteriorating weather conditions at Richmond obtained 
from the ATIS, Richmond Approach Control, and the display. He proceeded instead to 
the Wallops Island airport, initiating the new route just past the Hopewell VOR at 1925Z. 
The NEXRAD mosaic image that subject #7 was looking at on his weather information 
display as he made the decision to proceed directly to the Wallops Island airport is pro- 
vided in Figure 5.2-2. Note that the time stamp on this image was 19:14Z. Note also that 
the coded METAR report is indicating VFR conditions at the Richmond airport. 

Upon notification of convective weather ahead as he passed the Harcum VOR, subject 
pilot #7 worked out a new route with ATC to the south around the convective weather 
and then proceeded safely to the Wallops Island airport. The NEXRAD mosaic image 
that subject #7 was using as he contemplated how to avoid the convective weather ahead 
of him is provided in Figure 5.2.3. The display would have updated to the image provided 
in Figure 5.2.4 shortly after subject #7 turned to the south to avoid the hazardous weather. 
Upon receipt of this update, and subsequent updates, subject #7 had to continue to re- 
evaluate the proximity of his flight path to the hazardous weather as he proceeded to his 
destination. 

The ground tracks of all of the subject pilots are provided in Appendix P, Ground Track 
of All Subject Pilots. The full set of 1 1 NEXRAD mosaic images for the time frame en- 
compassing the mission scenario for all subjects is provided in Appendix Q, NEXRAD 
Mosaic Images. 
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Figure 5.2-1. Ground Track of Subject Pilot #7 



Figure 5.2-2. 19:14Z Display NEXRAD Mosaic Image 
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Figure 5.2-3. 19:21Z Display NEXRAD Mosaic Image 



Figure 5.2-4. 19:28Z Display NEXRAD Mosaic Image 
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5.3 Overview of Richmond and Wallops Decisions 


The results of the experiment have been organized around the two key decision points 
established in the experiment procedure - the “Richmond Decision” and the “Wallops 
Decision.” 

The “Richmond Decision” required the subject pilot to decide whether or not to attempt 
to land at the Richmond airport in the face of a fairly rapidly moving thunderstorm pass- 
ing within a mile or two at most to the north of the airport. There were a total of 1 1 dif- 
ferent NEXRAD mosaic images displayed to the pilot, updating in 7-minute intervals. 
Figure 5.2-2 depicts the NEXRAD mosaic image seen by the pilot upon arrival in the vi- 
cinity of the Final Approach Fix to the southeast of the Richmond airport. Because of the 
delay in transmission of the image to the aircraft, the data is about 7 minutes stale. Actual 
conditions at the Richmond airport in the time frame of this decision can be seen in Fig- 
ure 5.2.3 which provides the NEXRAD image for 19:21Z. The images depicted in these 
figures were the 4 th and 5 th images in the sequence of 1 1 images used. The thunderstorm 
seen to the northwest of Richmond is the storm that was designed to elicit a weather deci- 
sion from the pilots. The actual recorded storm was just a little smaller than the one de- 
picted to the pilot (was enlarged with photo retouching software). This particular storm 
moved from west to east across the successive NEXRAD images at approximately 40 
nautical miles per hour in the early images. The rate of movement of the storm dimin- 
ished to less than 10 nautical miles per hour in the later images. 

A good decision was deemed to be one in which the pilot decided to divert to Wallops 
prior to the Final Approach Fix of the approach into the Richmond airport so as to avoid 
the hazardous weather by at least five nautical miles. A poor decision was deemed to be 
one in which the pilot continued with an approach past the Final Approach Fix into the 
Richmond airport for whatever reason, placing the aircraft within five nautical miles of 
hazardous weather conditions. Flazardous weather was established to be a red NEXRAD 
mosaic image cell, a known area of hazardous turbulence, or a known area of hazardous 
windshear. A minimum separation of 5 miles from the most hazardous part of convective 
weather depicted in a NEXRAD image (red cells) was selected as the criteria for this 
segment of the scenario because: 

a. The hazard is a rapidly moving and fairly localized thunderstorm with a well-defined 
leading edge, 

b. The weather condition five miles and greater to the east of the thunderstorm in this 
actual weather data set were known to be reasonably safe with no significant turbu- 
lence, and 

c. The motivation created by the medical scenario to proceed to within a reasonable but 
safe distance. 

Table 5.3-1 provides a summary of the results for all subject pilots in the control group 
(without display) for this decision. Table 5.3.2 provides a summary of the results for all 
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subject pilots in the treatment group (having the display) for this decision. In addition to 
tabulating the decisions of each subject pilot, this table indicates the sources of weather 
information the pilots used to make this decision, the separation between the aircraft and 
the weather hazard, and the major factors influencing the pilots’ decisions. 

Six of the twelve control group pilots (without the weather information display) were 
judged to have made good decisions at Richmond. Six of the thirteen treatment group 
pilots were judged to have made good decisions at Richmond. 

The “Wallops Decision” required the subject pilot to decide whether to proceed directly 
to Wallops or detour around the hazardous weather. To proceed directly required the pilot 
to find his or her way between the thunderstorms located between the aircraft and the 
Wallops destination. Figure 5.3-1 provides the approximate image that the pilots 
equipped with the weather information display would have seen after departing the vicin- 
ity of the Richmond airport and upon notification by ATC of weather ahead as they 
passed the Harcum VOR (about 25 miles to the east of the Richmond airport). Figure 5.3- 
2 provides the NEXRAD mosaic image more correctly representing the actual conditions 
in this time frame. In these images, there is a line of convective activity over the Chesa- 
peake Bay and between Richmond and the Wallops Island airport. Within this line of 
convective activity are two thunderstorm cells that did not move significantly in position, 
but that changed shape and size slightly between images. These two thunderstorm cells 
were enhanced slightly with photo retouching software to create an enticing corridor that 
tempted pilots to fly between them. 

A good decision was deemed to be one in which the pilot, upon notification by ATC of 
weather ahead, circumvented the hazardous area entirely by rerouting to the south around 
the area of convective activity so as to avoid it by at least ten nautical miles, and then 
proceeding up the coast of Virginia to the Wallops airport. A poor decision was deemed 
to be one in which the pilot decided to find his or her way between the thunderstorms in 
an attempt to proceed by the most direct route to the Wallops airport, and for whatever 
reason, coming with ten nautical miles of hazardous weather. An attempt to reroute to the 
north around the convective activity was also deemed to be a poor decision as the con- 
vective area continued into extensive restricted airspace that was in use and not available 
to the subject pilot. Table 5.3-3 provides a summary of the results for all subject pilots in 
the control group (without display) for this decision. Table 5.3-4 provides a summary of 
the results for all subject pilots in the treatment group (having the weather information 
display) for this decision. In addition to tabulating the decisions of each subject pilot, 
these tables indicate the sources of weather information the pilots used to make this deci- 
sion, and the major factors influencing the pilots’ decisions. 

Eleven of the twelve control group pilots (without the weather information display) were 
judged to have made good decisions enroute to Wallops. Five of the twelve treatment 
group pilots (with the weather information display) that proceeded to Wallops were 
judged to have made a good decision enroute. 
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Table 5.3-1. Richmond Decision (Subjects Without Weather Display) 


Comments 

Ignored ATC info re hazards 

Aborted Flight due to WX 



Left ATC freq. @ OM to check ATIS 
(too late) 

Plamred to “take a look” 


Planned to “take a look” 

Seemed unconcerned about hazards 

Seemed unconcerned about WX 




WX Sources Used for Avoidance 

FSS/ 

FLT 

WATCH 


X 






1 




1 

PHF 

DEP 


1 






X 




X 

ATC 

X 

X 

X 

X 




X 




X 

ATIS 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Decision 

Poor 

X 

1 

1 

1 

X 

X 

■ 

X 

X 

X 

1 

1 

'O 

Good 

■ 

X 

X 

X 

■ 

1 

X 

1 

1 

1 

X 

X 

'O 

Reason for Diverting 

ATC 

Direction 

X 




X 

X 


X 

X 

X 



Total 

Own 

deci- 

sion 


X 

X 

X 



X 




X 

X 

Concerned re 
departing 
Richmond 



X 








X 


Prox to 
Red 
Cell 
(NM) 

2 (while 
holding) 

in 

<N 

in 

m 

O 

04 


co 

<N 

04 

cn 

- 


kO 

cn 

Csl 

Divert 
Distance 
(NM from 
RIC) 

5 (OM) 

o 

04 

o 

m 

04 

04 

5 (OM) 

5 (OM) 

co 

04 

5 (OM) 

5 (OM) 

5 (OM) 

O' 

cn 

04 

WX 

Display 


Z 

z 

Z 

Z 

Z 

Z 

Z 

Z 

Z 

z 

Z 

Pilot 

# 


00 

m 

(N 

O' 


in 

<n 

Os 

cn 

00 

in 

cn 

cn 

cn 

in 


'OVI” = Outer Marker 




























































































































Table 5.3-2. Richmond Decision (Subjects With Weather Di 


> 

ro 

a 

(A 


Comments 


Wanted to beat 
TSTM into RIC 


Delayed turn to 
East 








Over-used 
METAR function 



Own- 

ship 

Posi- 

tion 

Uncer- 

tain 

XI 

1 


X 


X 

X 

X 

X 

X 


X 


Dis- 

tance 

Estima- 

tion 

Error 

X 

X 





X 

X 

X 

X 

X 



Dis- 
regarded 
NEXRAD 
Data De- 
lay 

X 

X 

X 





X 

X 

X 




WX Sources Used for Avoidance 

FSS/ 

FLT 

WATCH 














PHF 

DEP 














ATC 

X 


X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

ATIS 

X 

X 

X 



X 



X 

X 

X 

X 

X 

Decision 

Poor 

1 

X 

X 



1 


X 

X 

X 

X 

X 

1 

r- 

Good 

X 

■ 

1 

X 

X 

X 

X 

1 

1 


1 


X 

NO 

Reason for Diverting 

ATC 

Direc- 

tion 


X 

X 

1 




X 

X 


X 



o 

H 

Own 

deci- 

sion 

X 



X 

X 

X 

X 



X 


X 

X 

Concerned re 
departing 
Richmond 

X 



X 

X 







X 


Prox. 
to Red 
Cell 
(NM) 

o 

CN 

co 

co 

IT) 

to 

to 

<N 

o 

CO 

CO 



CO 

CO 

O 

Divert 

0 0 

1 5 _ 
3*0 

5 S 2 

2 5- 

CN 

5 (OM) 

I 5 (OM) 

to 

! 5 (OM) 

CO 

CO 

<N 

CO 

1 5 (OM) 

! 5 (OM) 

r-» 

1 5 (OM) 

5 (OM) 


WX 

Display 

X 

X 

X 

X 

X 

X 


>< 



X 

X 


Pilot 
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co 

<N 

O 

co 

o 

CN 

CO 

o 

to 

00 

CO 

r- 

to 

<N 

00 

ON 

-3- 

to 

NO 

r-~ 


'OVI” = Outer Marker 


























































































































































Figure 5.3-1. 19:21Z Display NEXRAD Mosaic Image 









Figure 5.3-2. 19:28Z Display NEXRAD Mosaic Image 







Table 5.3-3. Wallops Decision (Subjects Without Weather Display) 


Comments 






To much time on FSS Freq. 







Too much time on Flight 
watch. Ignored ATC. 


Delay in Com- 
municating with 
ATC 






X 








WX Sources Used for Avoidance 1 

ATIS 


PHF/RIC/ PHF 











Flight 

Watch 


X 



X 




X 

X 

® 

® 

FSS 


1 


X 

1 



X 

1 

1 

X 


ATC 




® 

® 

® 

® 

® 

® 

® 



Decision 

Poor 


1 

1 

1 

1 

i 

1 

1 

1 

1 


X 

fl 

Good 

X 

fN 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


- 

Prox. to 
Red Cell 
(NM) 

O 

NA 

o 

O 

• i 

oo 

O 

O 

» < 

O 

» < 

O 

o 

'O 


Total 

WX Dis- 
play 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Pilot # 

Un 

00 

cn 

CM 

r- 

rr 


un 

G\ 

cn 

00 
r— H 


cn 

cn 

cn 

un 


CN 


Primary source for decision is ® . 

Enroute to Richmond, decided weather too hazardous to continue mission, returned to PHF. 























































































Table 5.3-4. Wallops Decision (Subjects With Weather Display] 

Prox. to Dis-regarded 

Pilot WX Red Decision WX Sources Used for Avoidance 1 NEXRAD Misinter- Incorrectly Delay in 

# Dienlnv fell i _ — | — I — 1 — — I — | 77777 Data Delay preted Used WX Coxmrni- Comments 





3 S 

- 


0 

<N 

T— H 

r- 

<N 

0 

T— H 


VO 


- 

1 

0 

2 

0 

5 




>" 



>" 







f. 

cn 

0 

O 

cn 

0 

00 


CN 

00 

G\ 


VO 



CN 

m 

<N 

IT) 

cn 



Tf 






CN 


Total 5 7 

Primary source for decision is D . 

Misunderstood mission scenario. Did not proceed to Wallops. Data not included in Wallops decision. 




































































































































One consequence of the selection process was an even distribution of the subject pilots 
between the control and treatment groups with respect to number of flight hours in actual 
instrument conditions. Tables 5.3-5 and 5.3-6 provide a summary of the total flight hours 
and actual instrument flight hours as reported by the subject pilots for the control group 
(no display) and the treatment group (with display) respectively. These tables also tabu- 
late the workload of the pilots over the entire mission as it was perceived by the pilots 
themselves and as it was observed by the experiment observers. Finally, these tables 
tabulate the observers’ perception of the overall proficiency of the subject pilots in flight 
in simulated instrument meteorological conditions as they performed the mission. The 
average number of actual instrument flight hours of the subject pilots was 135 hours in 
the control group and 151 hours in the treatment group. The number ranged from a low of 
4 hours to a high of 340 hours in the control group, and from a low of 5 hours to a high of 
500 hours in the treatment group. 

Table 5.3-5. Pilot Performance/Workload and Flight Hours (Control Group) 


Pilot# 

Total 

Flight 

Hours 

Act. Instrument 
Flight 
Hours 

Pilot 

Reported 

Workload 

Observed 

Workload 

Observed 
Pilot IMC 
Proficiency 







54 

700 

250 

low 

low 

high 

47 

400 

13 

high 

high 

low 

53 

1015 

80 

high 

moderate 

moderate 

8 

725 

70 

high 

moderate 

low 

5 

670 

15 

high 

high 

low 

33 

700 

80 

high 

high 

low 

55 

940 

15 

moderate 

moderate 

high 

44 

1815 

340 

low 

low 

high 

43 

2000 

230 

moderate 

low 

high 

39 

5200 

275 

low 

low 

high 

24 

4100 

250 

low 

low 

high 

18 

250 

4 

high 

high 

low 





Average: 

1542.9 

135.2 

Standard 

deviation: 

1557.0 

123.7 
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Table 5.3-6. Pilot Performance/Workload and Flight Hours 


(Treatmenl 

t Group) 


Total 

Act. Instrument 

Pilot 


Observed 

Pilot# 

Flight 

Flight 

Reported 

Observed 

Pilot IMC 


Hours 

Hours 

Workload 

Workload 

Proficiency 

57 

690 

25 

moderate 

moderate 

high 

50 

1800 

200 

moderate 

moderate 

high 

49 

1856 

200 

high 

high 

low 

48 

350 

15 

high 

high 

low 

45 

3000 

300 

moderate 

moderate 

high 

42 

900 

5 

very high 

very high 

very low 

30 

1000 

200 

high 

high 

moderate 

23 

1070 

50 

high 

high 

very low 

20 

800 

50 

high 

very high 

very low 

7 

1500 

150 

high 

moderate 

high 

3 

5600 

250 

high 

high 

moderate 

6 

2500 

500 

moderate 

moderate 

moderate 

38 

1000 

20 

high 

high 

low 







Average: 

1697.4 

151.2 




Standard 

deviation: 

1393.0 

145.6 





5.4 Results of Immediate Reaction Questionnaire 

Upon completion of the simulator session, each subject pilot was given a questionnaire 
(Appendix J, Immediate Reactions Questionnaire) to obtain their immediate reactions. 

The pilot was given this questionnaire while still seated in the simulator, thereby reducing 
distraction issues and obtaining valuable subjective information while the pilot was still 
in the “flight mode.” Some questions pertained to the use of the weather information dis- 
play, so the pilot of the control group without the display received a scaled down ques- 
tionnaire. 

Except for the last two questions, there were five available answers that ranged from Dis- 
agree (score of 1) through No Opinion to Agree (score of 5). This questionnaire was re- 
viewed with the pilot during the post- flight briefing to verify that the pilot understood the 
questions and to clarify any ambiguous answers. 

Question la . Pilots using the weather information display were asked if an ownship 
symbol would have been useful. (The prototype unit used in the experiment did not fea- 
ture ownship symbology.) 

An “ownship” aircraft symbol, cross-hairs or some similar position indica- 
tion would be a useful addition to the weather display. 

( mean score of 5.0) 

All of the respondents circled the Agree selection. 
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Question lb . Pilots using the weather information display were asked if the absence of 
an indication of ownship position on the display compromised the usefulness of the 
weather display. 

Without some way to ascertain position relative to other displayed features, 
the display’s usefulness is significantly compromised. 

(mean score of 4.15) 

Questions 2a, 2b. & 3 . These questions were primarily to determine the extent to which 
the subject pilots “bought into” the medical scenario presented to the pilots and were 
given to all the subject pilots. Essentially, the same question was asked three different 
ways to gain concurrence and validity. 

I took the medical emergency scenario seriously, in the sense that I factored 
the emergency into my decision making. 

(mean score of 4.24) 

While taking the medical aspect of the flight seriously, it did not figure at the 
forefront of my in-flight decision making. 

(mean score of 2.84) 

My knowledge that this was all a simulation, that nobody’s health or welfare 
was really at stake, influenced the way in which I managed the flight. 

( mean score of 1.68) 

Question 4 . This question explored if the treatment group pilots’ perception of the extent 
to which the weather information display depicted weather in “real-time.” 

An advantage of the onboard Weather Display was showing the weather in 
real-time, that is, as it actually was at that moment. 

(mean score of 4.38) 

During the post-flight interview, some of the pilots were aware that the NEXRAD images 
were up to 14 minutes old. Others stated that 14 minutes old is real-time compared to pre- 
flight weather charts that could be over an hour old. 

Question 5 . This question explored the treatment group pilots’ perception of the degree 
to which they felt they were able to interpret the weather information display and the ex- 
tent to which the display influenced their decision making. 

I attribute much of my decision making to my interpretation of the Weather 
Display 

(mean score of 4.08) 
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Question 6 . This question asked all of the pilots their perception of the extent to which 
they used all available weather information sources. 

I tried to systematically sample all sources of weather information open to 
me. 

(mean score of 3.20) 

Question 7 . This question explored the treatment group pilots’ perceptions as to their 
willingness to depend solely on the weather information display without cross-checking 
with information from other sources. 

I used the Weather Display but felt the need to cross-check or verify my 
conclusions from conventional weather sources (ATC, etc.) 

(mean score of 3.38) 

Questions 8a and 8b . Two questions were asked of all the subject pilots about their com- 
fort with and reliance on the autopilot. 

I felt comfortable with the autopilot, in terms of understanding its use & op- 
eration. 

(mean score of 4.32) 

Without the autopilot my completion of the flight would have been com- 
promised. 

(mean score of 4.4) 

Most of the pilots felt comfortable with the autopilot and relied on its use to reduce the 
workload. This is also reflected by the fact that the autopilot was in use 83% of the time 
(for all the subject pilots). Many of the pilots stated that without the autopilot, they would 
have succumbed to an early termination of the flight. 

Question 10a . When asked about the validity of the weather information display, there 
were mixed reactions. 

The degree of validity of the weather data appearing on-screen was a factor 
I felt I held in mind as I flew. 

(mean score of 3.69) 

Question 10b . Pilots who used the weather information display were asked if they regu- 
larly referred to the timestamp of the weather information. 

I have been monitoring the weather display time stamp very regularly in 
my instrument scan. 

(mean score of 2.42) 
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Question 1 1 . All the pilots were asked what they perceived the weather conditions to be 
in the vicinity of the Richmond airport. 


At the time of my arrival at Richmond’s Airport, I knew that there 
was a storm... 

a. about 10 nm North West of the field. (1 selection) 

b. about 5 nm North West of the field. (2 selections) 

c. near the field. (9 selections) 

d. right at the field. (13 selections) 

The pilots that did not fly with the weather information display picked either answer C or 
D only (placing the storm close to the field), but the pilots that flew the weather informa- 
tion display were not certain of the actual distance and their responses varied. 

Question 12 . Pilots who flew with the weather information display were asked what they 
perceived the weather conditions to be on the route to Wallops Island. 


At the time I was en route to Wallops I saw, across my path of direct flight, 
what I took to be... 

a. penetrable storm. 

b. a navigable opening between convective cells. (6 selections) 

c. a non-navigable opening between cells. 

d. a wall of convective activity requiring diversion. (6 selections) 
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6 Analyses and Discussion 

Quantitative assessments were undertaken of the probable relationships between the 
weather display and the two key decisions around which the experiment was set up. 
Qualitative assessments based on the observations and expertise of the experiment team 
were also undertaken of significant issues related to the weather display that surfaced in 
the course of the experiment. 

6.1 Quantitative Assessment 

Some of the data collected in the experiment were purely nominal with zero representing 
“no” or “not good,” and 1 representing “yes” or “good.” The form of these data, there- 
fore, favored non-parametric tests, such as the Chi-Square test. This technique was used 
in testing the relationship of several variables to decision making adequacy, and provided 
a value for chi together with the probability (p) that a result was due purely to chance. A 
less than a one in twenty chance that a result was merely a random fluke (p< 0.05) is gen- 
erally regarded as statistically significant. This report adheres to that convention. Where 
the p value of a result does not meet that criterion (i.e. is “not significant”), does not, of 
course, imply that the result is without importance, or even that it is a mere fluke. Rather 
it flags the result as one that may be simple chance and, therefore, cannot be safely gener- 
alized to circumstances beyond the experiment from which it was derived. 

Other data, however, were continuous numeric data such as weather test scores, flight 
hours, questionnaire responses (a 1 to 5 Likert Scale), etc. The SPSS Inc. software pro- 
gram was used to analyze a data set containing all information collected in this experi- 
ment. Pearson Product Moment correlation coefficients were calculated to evaluate the 
relationships among the continuous numeric data (Table 6.1-1), and a regression analysis 
was conducted on the two decision points. 

6.1.1 Richmond and Wallops Decisions (with/without weather display) 

As described earlier in the Methods section, the experiment scenario featured two critical 
decision points, one on the approach to the airport at Richmond, and one en route to the 
Wallops Island airport, the briefed destination. These decisions were considered singly, 
independently, and together as a group. First, the Richmond decision was examined 
alone. Using the decision rating criteria set out earlier, 6 of the 12 pilots flying without a 
weather display made a good decision at Richmond (coded as 1 in the data file) and the 
remaining 6 made a decision rated as poor (and coded 0). Of those who flew with the 
weather display, 7 made decisions rated as good and 6 made decisions rated as poor. Pre- 
liminary inspection of these raw scores alone suggests little performance difference as a 
function of weather display presence or absence. A Chi-Square statistical test was used to 
examine the data and determine if there were any significant differences in expected and 
observed frequencies (for a discussion of the role of these in Chi-Square, see Heiman, 
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1992 5 ). In the case of the analysis of Richmond singly, the Chi-Square result was not sta- 
tistically significant — bearing out the initial appearance of the data. 

For the decision en route to the Wallops Island airport (hereafter referred to simply as the 
Wallops decision or decision point), it appears that better decisions were made in the ab- 
sence of the weather information display. 

With the weather display, 5 pilots made a good decision, and 7 made a poor decision. 
When there was no weather display installed, 1 1 made decisions rated as good and only a 
single pilot made a decision rated as poor. Again, a Chi-Square statistic test was used to 
examine the expected and observed frequencies at the Wallops decision point. In this case 
the differences in frequencies are statistically significant (%~ = 6.75, p < .05), supporting 
the conclusion that at the Wallops decision point, better decisions were made when there 
was no weather display. 


RICHMOND: 

Weather Display 

No Weather Display 

Good Decision 

6 pilots 

6 pilots 

Poor Decision 

7 

6 

WALLOPS: 

Weather Display 

No Weather Displav 

Good Decision 

5 pilots 

1 1 pilots 

Poor Decision 

7 

1 


6.1.2 Display vs. No Display for Both Decisions Together 

Additional analyses were conducted to examine the combination of decisions at the two 
decision points, depending on whether the pilots received a weather display or not. The 
following table displays the frequency of good and poor decisions at each of the decision 
points. The table shows the number of pilots who made a decision that was rated as good 
at both decision points, the number said to have made poor decisions at both decision 
points, and the number rated as having made a poor decision at one and a good decision 
at the other. 


Pilots Given Weather Display: 

Richmond Decision Poor 
Richmond Decision Good 


Wallops Decision 
Poor Good 

0 Pilots 6 Pilots 

1 5 


5 Heiman, G. (1992) Basic Statistics for the Behavioral Sciences, Boston: Houghton- 
Mifflin 
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Pilots Not Given Weather Display: Wallops Decision 

Poor Good 

Richmond Decision Poor 4 Pilots 2 Pilots 

Richmond Decision Good 3 3 

The data from the above table were analyzed with the Chi-Square technique to examine 
whether access to a weather display resulted in better decision making at the two decision 
points of Richmond and Wallops (considered together as a unit of performance). This 
Chi-Square result was not statistically significant. This result suggests that combined 
Richmond/Wallops decision performance was unaffected by the presence or absence of 
the weather display. 

In this context, however, one cautionary note needs to be expressed. In the statistical lit- 
erature, some authors suggest that the validity of the Chi-Square analysis is questionable 
for cell populations less than 5. As can be seen in the table above, the expected frequen- 
cies fell below 5 in six of the eight cells. 

6.1.3 Flight Hours, with/without Weather Display, and Decisions Enroute 

Flight experience is, of course, a potentially important variable in understanding the dy- 
namics of behavior in any experiment such as that reported here. One measure of experi- 
ence of the subject pilots collected in this experiment was the number of total flight 
hours. An analysis was conducted to examine the relationships between total flight hours, 
weather display condition (presence or absence of the weather display), and decision 
making. In order to examine whether the number of flight hours played a role in decision 
making, the Chi-Square test was used to compare decisions made at Richmond and at 
Wallops. 

RICHMOND: 


No Weather Display Richmond Decision 

Poor Good 

Under 1000 Flight Hours 4 Pilots 3 Pilots 

Over 1000 Flight Hours 2 3 

Weather Display Richmond Decision 

Poor Good 

Under 1000 Flight Hours 3 Pilots 2 Pilots 

Over 1000 Flight Hours 4 4 


This Chi-Square was not significant for the decision at Richmond. The number of total 
flight hours did not seem to interact with the weather display condition to affect the deci- 
sion quality. 
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WALLOPS: 


No Weather Display Wallops Decision 

Poor Good 

Under 1000 Flight Hours 0 Pilots 7 Pilots 

Over 1000 Flight Hours 1 4 

Weather Display Wallops Decision 

Poor Good 

Under 1000 Flight Hours 3 Pilots 2 Pilots 

Over 1000 Flight Hours 4 3 


The Chi-Square for the Wallops decision was also non-significant. Total flight hours 
again did not appear to interact with the weather display condition to affect decision 
quality. 

Even though the Chi-Square results were not significant with the number of cases run in 
this experiment, there does appear to be a discernible trend with regards to the decision 
making at Wallops. This suggests that less experienced pilots tended to make decisions 
rated as good when flying without the weather display. The possibility that this is a pure 
chance related to the characteristics of the pilots and the conditions of this study cannot 
be excluded. However, it is indicative of a potentially perilous trend that warrants the 
need for larger scale studies capable of building upon the exploratory analyses reported 
here in order to seek more definitive answers. 

6.1.4 Risk Aversion, WX Knowledge, Experience, and Decision Making 

A regression analysis was conducted to examine the influence of risk aversion, total flight 
time, weather knowledge, and presence or absence of weather display on decision ratings. 
For the Richmond decision, the Multiple R = .37, and R 2 = .14. This is a non-significant 
result. 

A regression analysis was also conducted for the Wallops decision. Multiple R = .60, and 
R" = .36, p =.06, the largest factor being the presence of the weather display. This is not 
significant by reference to our p=0.05 cut-off. Rather than a 1 in 20 chance of being a 
fluke, the result has about a 1 in 17 chance of being a fluke. It is, therefore, still unlikely 
to be mere happenstance. Traditionally, “nearly significant” results, though themselves 
properly unreportable, have been taken to suggest that a real effect may exist and might 
be identified in further studies, especially where larger subject numbers increase the 
power of the statistics to identify effects unambiguously. 

The results of a Pearson Product Moment correlation coefficient computation performed 
for the ten variables represented by continuous numeric data in the experiment is pro- 
vided in Table 6.1-1. The ten variables define the rows and columns of the table. Each 
table cell contains two entries. The top entry is the correlation coefficient, which is a 
measure of the tendency of the intersecting variables to occur together (not necessarily 
cause and effect). Values close to 1.0 indicate that a high value in one variable tends to 
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occur with a high value in the second variable. Likewise, values close to -1.0 are indica- 
tive that a high value in one variable tends to occur with a low value in the second vari- 
able. The second value contained in each table cell is the probability that the correlation 
coefficient computed is due to random chance. 

For example, the correlation between “Weather Knowledge” and “Combined Score” is a 
rather high value of 0.866, with a low probability that this result is due to chance. This 
result is expected since the candidate pilots were purposefully divided into groups such 
that high weather knowledge pilots were placed in the high combined score group, and 
low weather knowledge pilots were placed in the low combined score group. Conversely, 
high risk pilots were also placed in the low combined score group; note the negative cor- 
relation coefficient of -0.8688 with a p value approaching 0, indicating a high probability 
that this result is statistically significant. 

In addition to the data based on the 10 continuous variables measured on the population 
of 25 pilots and shown in Table 6.1-1, a database consisting of some 41 variables meas- 
ured or observed on the treatment pilot group was recorded and archived using the SPSS 
software. The 41 variables are listed in Table 6.1-2. A database of the variables in Table 
6.1-2 is available from the RTI Flight Systems Engineering Office for further analysis. A 
far more comprehensive set of data was collected in the experiment for each subject in 
the form of simulation control inputs, flight path performance, observed data, and pilot 
debriefing records, and pilot questionnaires. This data is also available from RTI for fur- 
ther analysis. 

Taken in conjunction with the Chi-Square results, the best predictor of decision rating for 
the Wallops decision was whether the individual was flying with a weather display or 
not. The direction of the relationship might be considered disappointing, since the pres- 
ence of the display may be associated with poorer rather than better decision making. 

The results of the quantitative assessments presented here are consistent with some of the 
fears that have been expressed regarding undue optimism about the likely performance 
effects of the first generation of cockpit weather display systems. The sources of per- 
formance difficulties observed are complex and varied, however, and substantial infor- 
mation was also found in qualitative assessments of the performance of the subject pilots. 
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Table 6.1-1. Pearson Product Correlation Coefficient Computation Results 



Risk 

Score 

Weather 

Score 

Combined 

Score 

Flight 

Hours 

Question 

#2A 

Question 

#2B 

Question 

#3 

Question 

#8A 

Question 

#8B 

Question 

#11 

Risk 

Score 




0.2110 

0.156 

-0.0890 

0.336 

-0.0362 

0.432 

-0.1664 

0.213 

0.1583 

0.225 

0.1416 

0.250 

-0.2786 

0.089 

Weather 

Score 


■ 

0.8660 

< 0.000 

H 

0.1529 

0.233 

0.1494 

0.238 

0.3592 

0.039 

-0.2756 

0.091 

-0.2465 

0.117 

-0.0211 

0.460 

Comb. 

Score 



■{ 

0.0578 

0.392 

0.1400 

0.252 

0.1064 

0.306 

0.3024 

0.071 

-0.2505 

0.114 

-0.2232 

0.142 

0.1489 

0.239 

Flight 

Hours 





-0.0820 

0.348 

M 

-0.2029 

0.165 

-0.2473 

0.117 

-0.1560 

0.228 

-0.2198 

0.146 

Question 

#2A 






-0.6434 

<0.000 

-0.0031 

0.494 

-0.1239 

0.278 

-0.1248 

0.276 

-0.2432 

0.121 

Question 

#2B 



corr coeff 
p value 




0.2105 

0.156 

0.1282 

0.271 

0.1968 

0.173 

-0.0186 

0.465 

Question 

#3 



(typical) 





0.3542 

0.041 

H 

0.0458 

0.414 

Question 

#8A 








IKf 

0.1690 

0.210 

-0.0503 

0.406 

Question 

#8B 











Question 

#11 










| 


-1 .0 < correlation coefficient < 1 .0 

p = probability that calculated correlation coefficient is due to chance. 
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Table 6.1-2. Measured or Observed Experiment Variables 

1 Richmond decision 

2 Wallops decision 

3 Risk Score 

4 Weather Score 

5 Combined Score 

6 Flight Hours 

7 Answer to Question 1 a I 

8 Qlb 

9 Q2a 

10 Q2b 

11 Q3 

12 Q4 

13 Q5 

14 Q6 

15 Q7 

16 Q8a 

17 Q8b 

18 QlOa 

19 QlOb 

20 Q 1 1 

21 Q12 

22 Pilot Unaware of data stale @ Richmond 

23 Pilot Unable to estimate distances @ Richmond 

24 Pilot Unable to determine ownship location @ Richmond 

25 Pilot reported Richmond departure concerns 

26 Pilot waved-off by ATC @ Richmond 

27 Pilot used Richmond weather source: AT1S 

28 Pilot used Richmond weather source: ATC 

29 Pilot used Richmond weather source: FSS 

30 Pilot used Richmond weather source: FlightWatch 

3 1 Pilot used Richmond weather source: METAR text 

32 Pilot used Richmond weather source: WX display 

33 Pilot Disregarded NEXRAD @ Wallops 

34 Pilot Misinterpreted display @ Wallops 

35 Pilot Used display for Navigation @ Wallops 

36 Communication delay incurred enroute to Wallops 

37 Pilot used Wallops weather source : ATC 

38 Pilot used Wallops weather source : FSS 

39 Pilot used Wallops weather source : FlightWatch 

40 Pilot used Wallops weather source : METAR text 

41 Pilot used Wallops weather source : WX display 


Immediate Response 
Questionnaire Answers 
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6.2 Qualitative Assessment Results 

The difficulties in the use of the weather information display suggested in the quantitative 
analysis appear to include workload problems, incorrect assumptions about the accuracy 
and timeliness of displayed weather data, misuse of the display as a navigation aid and 
failure to cross-check information from other available sources. 

While it was required that all candidate subject pilots for this experiment be qualified and 
current as instrument pilots, the 25 pilots ultimately selected to participate demonstrated a 
very wide range of performance in instrument flight. Their selection from the pool of 
candidates was based on their scores on the risk aversion test and weather knowledge test 
and not on their total flight hours nor on their number of flight hours in actual instrument 
conditions. Their proficiency in instrument flight operations was very probably quite rep- 
resentative of the population of general aviation pilots having similar qualifications and 
levels of experience. 

As indicated in Tables 5.3-5 and 5.3-6, there was remarkably good correlation between 
demonstrated instrument flight proficiency (at least as observed in the simulation facility 
used in the experiment) and actual instrument flight time. The five pilots within the con- 
trol group who stated they had 250 or more hours of actual flight time in instrument con- 
ditions were observed to have no significant problems in performing the mission. The 
other seven pilots with less actual instrument flight time were all observed to be less pro- 
ficient and experienced a significantly higher workload in performing the mission. 

Among these seven less experienced pilots, the most experienced in actual instrument 
flight reported he had 80 hours of actual instrument flight time while the least experi- 
enced reported he had 4 hours of actual instrument time. 

Within the treatment group, there was a similar trend, although the correlation was not as 
pronounced. Six of these pilots reported they had 200 or more hours of actual instrument 
flight time; five of the six were observed to have no significant problems in performing 
the experiment mission. The sixth pilot reported that he had flown most of his 200 hours 
of actual instrument time several years before in the military, and had only flown a few 
hours of actual instrument time in the past several years. Among the six subject pilots in 
the treatment group who reported they had less than 200 hours of actual instrument flight 
time, all six were less proficient and experienced a significantly higher workload in per- 
forming the experiment mission. 

Despite the observed instrument flight experience/flight proficiency relationship, there 
was no apparent correlation between the actual instrument flight time of the subject pilots 
and the decisions they made at the two decision points in the experiment. Among the six 
pilots within the treatment group with 200 or more hours of instrument flight time, only 
three made good decisions at the Richmond decision point and only three made good de- 
cisions at the Wallops decision point. 

Nor does there appear to be any correlation between the actual instrument flight time of 
the subject pilots in the treatment group and their success in use of the weather infomia- 
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tion display. Among the five pilots within the control group with 250 or more hours of 
actual instrument flight time, only two made good decisions at the Richmond decision 
point. 

Of the 12 subject pilots in the control group, five made good decisions at both the Rich- 
mond and Wallops decision points. Three of these pilots were observed to be highly pro- 
ficient in instrument flight and experienced a relatively low level of workload; two were 
observed to be low in proficiency in instrument flight and experienced a relatively high 
level of workload. 

Of the 13 subject pilots in the treatment group (had weather display), only two made 
good decisions at both the Richmond and Wallops decision points. One of these pilots 
was observed to be highly proficient in instrument flight and to have experienced a rela- 
tively low level of workload; the other was observed to be low in proficiency in instru- 
ment flight and to have experienced a relatively high level of workload. 

6.2.1 Workload Issues 

This study was not designed to specifically measure or quantify workload in relation to 
the use of the weather information display, however, general observations were made by 
the observers who participated in the experiment. The observers gathered workload cues 
such as incorrect or inappropriate procedures, fixation, body movements, hesitation in 
communication, changes in voice pitch, control excursions, flight technical errors, proce- 
dural hesitation, navigation errors, haphazard search techniques, autopilot use, training 
transfer problems and physiological cues such as perspiration. 

Observed workload, and self-perceived workload of the subject pilots participating in the 
experiment with and without the weather information display were reported as follows 
(see Tables 5.3-5 and 5.3-6); 

Pilots without Weather Display 

Observed Workload 
High 4 Pilots 

Moderate 5 

Low 3 


Pilots with Weather Display 

Observed Workload 
High 8 Pilots 

Moderate 5 

Low 0 

The subject pilots in the treatment group (with weather display) were asked if they felt 

the weather display had an impact on their workload. Seven of the 13 subject pilots with 
the weather display reported that they felt the display increased their workload. Six re- 
ported that they felt the display decreased their workload. 


Self-perceived Workload 
9 Pilots 
4 
0 


Self-perceived Workload 
6 Pilots 
2 
4 
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As indicated previously, the overall workload observed by the experimenters as well as 
the overall workload perceived by the pilots themselves appeared to be relatively higher 
overall for the treatment group than for the control group. The lowest level of observed 
workload in the treatment group was reported to be a moderate level for 5 pilots whereas 
in the control group 5 pilots were reported to have experienced a low level of workload. 

The observed workload and the self-perceived workload were low for the subject pilots in 
the control group reporting over 250 hours of flight in actual instrument conditions. The 
observed workload and the self-perceived workload were relatively higher for the subject 
pilots in the treatment group reporting over 200 hours; four of the six experienced a rela- 
tively moderate workload and the other two experienced a relatively high workload. 

The subject pilots were asked to comment on the workload for the entire mission, and 
were also asked what the workload would have been if an autopilot had not been avail- 
able. All but one of the subject pilots had experience in the use of autopilots typical of the 
one implemented in the experiment; all recognized the autopilot to be an important asset 
to proficient instrument flight operations. All the pilots had an autopilot available during 
the experiment and were trained on its use during the training session. During the pre- 
mission briefing, the pilots were instructed to use the autopilot if they felt it necessary to 
do so, but there was neither any requirement nor penalty in its use. Figure 6.2. 1-1 depicts 
the extent to which the subject pilots used the autopilot in the experiment. 



Subject Pilot 


Figure 6.2.1 -1. Percent of Flight Time Autopilot Used 


Because of their relatively low level of experience and currency in actual instrument 
flight, the workload was relatively high for approximately one half of the subject pilots. 
Even though the autopilot was used for an average of 82% of the time in flight across all 
the subject pilots, some were too busy to effectively integrate the use of the weather in- 
formation display into their procedures. Others were able to effectively use only one or 
two functions of the display. All of the pilots in both groups stated the autopilot was ei- 
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ther essential to the safe accomplishment of the flight or substantially reduced the work- 
load of flying in instrument conditions. Nearly all of the pilots in the treatment group 
stated that in reducing their workload, the autopilot made it possible for them to make 
more effective use of the weather information display. 

6.2.2 Use of Available Weather Data Sources 

The source of weather information of choice for the approach into the Richmond airport 
was the Richmond airport ATIS. All of the pilots without the weather display used the 
ATIS while three fourths of the pilots with the weather display also used the ATIS. One 
half of the control group pilots asked Richmond Approach Control for the latest condi- 
tions of the field while all but two of the treatment group pilots asked ATC for the latest 
conditions. The majority of the pilots having the weather infomiation display queried the 
textual METARs. There were only two cases, however, of pilots using the Flight Service 
Station or Flight Watch to update their knowledge of weather conditions enroute to 
Richmond (one control group pilot and one treatment group pilot). 

While enroute from Richmond to the Wallops Island airport, all but one of the subject 
pilots without the weather display depended on ATC for information and guidance 
around the hazardous weather conditions. The one pilot who did not depend on ATC for 
guidance based his navigation decisions on information received from the Flight Service 
Station and Flight Watch. The subject pilots having the weather display, however, almost 
totally ignored all other sources of weather information except ATC. Eight out of the 
twelve pilots with the weather information display used the display as their primary 
source of information for avoiding the hazardous weather while consulting with ATC. 
Four of the pilots with the weather information display depended primarily on ATC for 
avoidance of the hazardous weather while consulting the weather display. 

In several cases, pilots with the display made the strategic decision to proceed around the 
thunderstorm activity without help from ATC even though they had been alerted by ATC. 
It was interesting that when asked in the post-flight debriefing about their use of in-flight 
weather advisories from the Flight Service Station, the subject pilots having the weather 
information display commented that they do not use the service much because the verbal 
infomiation is difficult to interpret and takes an excessive amount of time to collect. Ad- 
ditionally, the FSS does not generally know their specific location and obtaining specific 
route infomiation is difficult. Yet, their counterparts in the control group who did not 
have the display used these sources effectively to develop an understanding of the 
weather conditions enroute to the Wallops Island airport. 

6.2.3 Interpretation of the Weather Information Display 

Nearly all of the subject pilots in the treatment group were enthusiastic about the poten- 
tial for improving their awareness of weather conditions, but many misinterpreted or did 
not access the infomiation available from the display. Many of the poor weather deci- 
sions were made because the pilots, even though they had the weather infomiation dis- 
play, were not aware of the deteriorating weather conditions and were consequently sur- 
prised that they could not get into the Richmond airport or that there were thunderstorms 
over the bay between Richmond and the Wallops Island airport. Nearly all recognized 
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that a thunderstorm was in the vicinity of the Richmond airport - most failed to correctly 
recognize the level of the hazard to them. When asked why they proceeded with the ap- 
proach inside of the outer marker, they said they had not recognized how close they were 
getting to the hazardous areas of the storm. Many said they had decided to proceed with 
one approach into the Richmond airport and “see what happens.” Most of these pilots 
said they were looking for clues as to the severity of the weather such as turbulence lev- 
els, lightning or level of rain. Only one subject pilot having the weather information dis- 
play noticed that the graphic METAR depiction of visibility and ceiling at the Richmond 
airport had changed from VFR to IFR conditions as a result of a special report. He no- 
ticed the change after he had made the decision to divert to Wallops. 

6.2.3.1 Judging Proximity to Hazardous Weather 

Many pilots had difficulty correctly determining their position in relation to the storm 
cells. Some incorrectly determined their own position, others incorrectly determined the 
distance from their location to the hazardous areas of the storm. When comparing the re- 
corded distance from the aircraft to a hazardous weather condition (a red cell), half of the 
pilots misjudged the distance by two to four times the perceived distance, and all of the 
pilots that had a disparity placed the hazard farther away than it actually was. 

Most of the pilots were familiar with and experienced in the use of moving map displays, 
and thus were accustomed to seeing their position portrayed on a display screen such as 
the one used in this experiment. However, the weather display used in this experiment did 
not feature an ownship icon or clear indication of range. Half of the pilots with the dis- 
play demonstrated situational awareness problems that were confirmed in the debriefing. 
The problems were related to the inability to determine ownship position on the display. 
The subject pilots stated they spent a disproportionate amount of time attempting to lo- 
cate their approximate position on the weather display screen - a behavior pattern that 
was confirmed by the experiment observers. 

Even after a generous amount of time devoted to training in the use of the free and lock 
modes 6 , the subject pilots still had difficulty with this feature. This confusion over the 
positioning modes caused some pilots great difficulty and contributed to the already high 
workload. Many occasions were noted of the subject pilot scrolling excessively and se- 
lecting the METAR icons to determine the position of the weather image in relation to 
recognizable landmarks or navigation aids. The lack of an ownship position icon was the 
primary complaint of the display during the post-flight interview. 

Compounding the problem of determining proximity to hazardous weather, a number of 
subject pilots experienced difficulty in estimating distances on the weather information 
display. Some pilots misinterpreted the display distance equals miles indication on the 


6 The display had two orientation modes, GPS free and GPS lock. GPS free allowed the user to freely 
scroll the NEXRAD image to any location, thus allowing the range to be set low (for higher storm cell 
resolution) while scrolling to locations not in the general location of the aircraft. When the GPS lock mode 
was selected, the NEXRAD image was locked into a position with the physical center of the display be- 
coming the aircraft position in relation to the weather image. In this mode, the weather image moved with 
changes in aircraft position and use of the joystick to move the image was disabled. 
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display, thinking the number was an indication of the scale selected and indicated the 
number of miles from one side of the display screen to the other, or the number of miles 
from the bottom of the display screen to the top of the screen. Others understood the 
meaning of the distance scale, but felt they were unable to effectively use this indicator to 
estimate distance. Many subject pilots suggested that some form of range rings or other 
similar indication should be available to aid in estimating distances. 

In one instance, the subject pilot was making strategic and tactical decisions believing 
that the display had a track-up orientation instead of the actual north-up orientation, and 
completely lost confidence in his ability to use the display when his interpretation of the 
display disagreed with what ATC was telling him. 

6.2.3.2 Recognizing and Interpreting Effects of Delay 

The subject pilots with the weather information display were briefed twice (during the 
introduction and again during the familiarization flight) that the NEXRAD image could 
be from 7 to 14 minutes old and to check the image timestamp with the onboard clock. 
They were also apprised that the METAR information could be as much as an hour old 
and to check the issue time. Due to the lack of an ocular eye tracker, empirical data for 
how the subject pilots used the timestamp information was not available. Insight into how 
they used the timestamp information was only available through the interview process. 

While most of the subject pilots were aware of the delays in the display of the NEXRAD 
mosaic images, many either forgot the delays or chose to ignore them because of their 
workload. A few chose to assume some sort of average delay in the display of the image 
and planned to account for it; most of these pilots incorrectly estimated the impact of the 
delay in determining the movement of the hazardous weather and their proximity to it, 
especially near the rapidly moving front at the Richmond airport. 

Many of the subject pilots commented that they perceived the NEXRAD weather image 
to be real time information (without delay). When asked if they were aware of the age of 
the image, they generally commented that they were aware of the 7 to 14 minute delay at 
the beginning of the flight, but that they soon started to treat it as real time. Two pilots 
even commented that, to them, 7 to 14 minutes is real time compared to a preflight 
weather chart that could be hours old. When asked about how they used the timestamp 
information, most commented that they did not consistently determine the age of the 
NEXRAD image, but either ignored the delay or treated the image as delayed a consistent 
amount and did not try to determine the age of the image. Further investigation revealed 
that while most subject pilots were aware of the delay, the workload would have been 
excessive if the age of the image were to be determined every time the display was que- 
ried. Many pilots also commented that they were using the NEXRAD images for short 
term as well as long term decisions. 

The graphical depiction of NEXRAD weather images will be available to pilots through 
paid subscriptions. The effects of aging of the NEXRAD mosaic image on the pilot’s un- 
derstanding of proximity to hazardous weather conditions is likely to become more sig- 
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nificant if FISDL service providers or users are allowed to update their NEXRAD image 
displays even less frequently than the 7 minute intervals used in this experiment. 

Many subject pilots noted that an age indicator would have reduced the mental workload 
of determining the age of the NEXRAD image. Some suggested that an indicator that 
changed size or shape in proportion to the age of the image would reduce the mental cal- 
culations required to compare the image timestamp to the current time. 

The subject pilots with the weather information display were briefed twice (during the 
introduction and again during the familiarization flight) that the METAR information 
could be as much as an hour old and to check the issue time. Eight of the 12 subject pilots 
having the weather information display accessed the METAR text data at least once in 
making the Richmond decision. Three accessed the METAR text data at least once in 
making the Wallops decision. Most of the pilots were not aware of the METAR age, and 
admitted that the METAR infomiation is not that timely and was generally not consulted 
in their decision making. Only one of the pilots having the weather information display 
noticed that the coded METAR report for the Richmond airport had changed during the 
flight, and acknowledged that he noticed the change only as he was leaving the vicinity of 
the Richmond airport enroute to the Wallops Island airport. 

The METAR textual information is presented in typical ICAO teletype codes and al- 
though identical in content to the information that will be broadcast free of charge, the 
interpretation of the codes in a high workload environment causes many errors. In this 
experiment, many of the subject pilots admitted to having difficulty interpreting the 
codes. Many errors were observed and excessive fixation times were observed when the 
pilots attempted to decode the METAR information. Other studies reflect similar find- 
ings. In a report about Mode-S datalink weather information, Rehmann (1995) notes, 
“Once the codes are learned for the test, they are promptly forgotten because the FSS 
briefer will decode them in the weather briefing. With the implementation of DUATS, 
those codes are once again needing interpretation and come as a shock to our systems.” It 
was noted in this experiment that the airline pilots who participated did not have as much 
difficulty in interpreting the reports as did the general aviation pilots. This was also a 
finding in Rehmann. Many of the pilots commented that the METARs would be more 
useful if they were displayed with their English translation, much as DUATS provides the 
English translation. Rehmann also found that English translations were less prone to er- 
ror. 


6.2.3.3 Use of Weather Information Display for Navigation 

Several pilots were observed to be using the weather display for more than just enhance- 
ment of their awareness of their situation with respect to potentially hazardous convective 
weather conditions. These pilots were attempting to augment their understanding of their 
position derived from the VOR navigation system, and to then navigate from that position 
to another location on the display image. In at least one case they were attempting to ac- 
tually navigate using the weather display in lieu of the VOR navigation system - with 
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disastrous results. In this case, the pilot became totally disoriented, unable to reestablish a 
workable mental model within which to navigate, and consequently decided to abandon 
the flight. 

The fundamental problem was that the pilots were required to integrate two mental im- 
ages of their location; mental images developed from and based on two entirely different 
reference systems. One mental image was based on the integration of data consisting of 
distance and bearings with respect to several selected navigation facilities with which 
they were familiar; the other was based on a mental image of distance and bearings with 
respect to a different set of features depicted graphically on the weather display image. 
Compounding the problem, all of the features needed for navigation were not available 
from the weather information display. This invariably substantially increased their 
workload. 
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7 Conclusions 


The objective of the weather information experiment was to investigate the potential for 
misuse of weather information, and thus provide guidance for the FAA. The successful 
completion and documentation of the lessons learned and the knowledge acquired during 
the conduct of this experiment has met the objective, while indicating directions of future 
research. 

The elements of the experiment design have been substantiated. These elements included 
the selection method for subject pilots, the prototype FISDL display system selection, the 
simulator fidelity (cockpit instrumentation, out-the-window scene generation, ATC 
communication environment, etc.), the “between-subjects” approach, the use and re- 
cording of actual weather, the pre-flight training, the adequacy of the output data ob- 
served and recorded, the content and technique for the post-flight debriefing, and cor- 
roboration of the qualitative expert assessments with quantitative results, 

The navigation decisions were designed into the experiment to test the experiment hy- 
pothesis: “delayed weather information datalinked to the cockpit display may lead to 
navigation decision errors.” When the two key decisions in the experiment were consid- 
ered jointly, the presence of the weather information display had no significant statistical 
impact on the outcome of the decisions made by the pilots. The purpose of introducing 
the FISDL display, however, is to reduce the number of poor decisions, which was not 
the case. 

When the two key decisions in the experiment were considered separately, the presence 
of the display had no signi ficant statistical impact on the Richmond decision; it did have 
a significant impact on the Wallops decision. A statistically significant greater number of 
pilots with the weather display made a poor navigation decision with respect to avoiding 
the hazardous weather enroute to Wallops. The Richmond decision emphasized the tem- 
poral hazards associated with time delayed weather information. The Wallops decision 
emphasized the potential spatial hazards associated with data linked weather displays. 

The significant issues having an impact on the outcome of the experiment are summa- 
rized below. 
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7.1 Weather Information Display Interpretation Issues 

The configuration of the weather information display system implemented in this study 
did not improve the decision making of the pilots using the display. Causes for this find- 
ing include: 

a. Pilots were unable to easily perceive their proximity to potentially hazardous con- 
vective weather conditions graphically depicted because of: 

• Difficulty in determining ownship position due to lack of ownship symbol and 
other features with which to construct and maintain a mental model of posi- 
tion, 

• Difficulty in estimating distances on the display, and 

• Substantial latency in the presentation of NEXRAD and METAR data, and 
interpretation of the effects of that latency. 

b. Pilots were unable to easily estimate the juxtaposition of their flight path with the 
path of graphically depicted hazardous convective weather conditions (NEXRAD 
mosaic images) because own path was not depicted on display and probable path 
of the hazardous weather was not depicted. 

c. Use of the weather information display apparently increased the workload for at 
least half of the pilots, decreasing the time available for decision making. 

d. Difficulty for many pilots in deciphering METAR text data. 

7.2 Weather Source Information Issues 

The weather information display limitations described above not withstanding, the dis- 
play of NEXRAD mosaic images substantially increased the pilots’ awareness of the 
general location of convective weather in their vicinity. The compelling nature of the dis- 
play of these images, however, caused some pilots to depend too heavily on the weather 
information display for their information regarding hazardous convective weather condi- 
tion. As a result, they failed to obtain other essential and corroborating information from 
other available sources. 

7.3 Training Issues 

Substantial training in the use of a weather information display system will be required to 
help pilots understand the limitations of a weather information display and its data, to re- 
duce the workload otherwise required to access and interpret weather information, and to 
enable the pilot to fully exploit the potential safety contributions of the display. 

7.4 Autopilot Issues 

The safe and effective use of a weather information display in actual instrument condi- 
tions will almost certainly require the support of an autopilot for most pilots. 
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8 Recommendations 


The following recommendations are based on the findings of this study, loosely format- 
ted for possible incorporation in the FAA Aeronautical Information Manual (AIM), draft 
FAA Advisory Circular No: 00-FIS, titled “Use of Cockpit Displays of Digital Weather 
and Operational Information,” and draft FAA Advisory Circular No: 20-FIS, titled 
“Safety and Interoperability Requirements for FIS Equipment.” The recommendations 
pertaining to the AIM are limited to information that does not duplicate information al- 
ready provided in the latest edition of the AIM (August 25, 2000). 

Additional recommendations are provided for the consideration of the FISDL display 
system manufacturers. 

8.1 AIM and Advisory Circular Recommendations 

The depiction of weather information, including NEXRAD and METAR products, will 
be delayed due to the time required for the collection and distribution of vast amounts of 
weather information available. 

The time required to produce the NEXRAD mosaic display includes a six-minute cycle 
for the individual NEXRAD radars to scan and observe the data. An additional interval is 
required for the automated processing of the NEXRAD data necessary to merge all the 
individual NEXRAD radar images into one national mosaic before the NEXRAD na- 
tional mosaic is available from which to create the FISDL cockpit images for transmis- 
sion. 

METAR observations are only produced once an hour. The hourly METAR observation 
remains as the "official" observation for the airport throughout the hour and is included in 
the airport ATIS. During approach the pilot will be provided the direct readout wind and 
altimeter information by the tower controller. In addition, pilots can obtain the aural re- 
port of the latest minute ASOS observation while in radio range of the airport ASOS. 
During dynamic, changing weather conditions, SPECI observations are issued and are 
included in new ATIS reports, but they are unscheduled and are thus unpredictable in 
terms of knowing or anticipating when they should be available. In addition, TAF fore- 
casts are issued four times per day at scheduled intervals and remain valid until amended 
or superceded by the next issued TAF. TAF AMEND, like SPECI observations, are un- 
scheduled and are thus also unpredictable in terms of when they should be available. The 
availability of SIGMET, AIRMET, PIREP and AWW reports is similarly unpredictable 
in that they are primarily event driven and issued (or amended) when weather conditions 
dictate. 

Another delay introduced into all the FISDL products is a product of the FISDL broad- 
cast transmission cycle. The communication architecture of the FISDL broadcast will 
determine the magnitude of that delay for any specific FISDL product. For example, the 
FISDL Service Provider may decide to place a priority on transmitting NEXRAD prod- 
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ucts and thus "interrupt" any text transmissions when a new NEXRAD product is re- 
ceived. 

It is essential that the pilot become fully proficient in determining and maintaining a 
comprehensive awareness of the age of each of the FISDL display weather information 
products so as to be able to effectively and accurately integrate this information 
(NEXRAD image time stamps, METAR text time data, etc.) with the information gath- 
ered from the other sources. 

Because of the inherent production delays, the weather information provided by a 
weather information display should not be used for avoiding hazardous weather in a tac- 
tical manner, such as finding one’s way through a line of thunderstorms. In the time that 
it takes for a NEXRAD image to be produced and transmitted, a storm cell could have 
moved a significant distance. Stomi cells can also sometimes develop very quickly to 
hazardous levels within the update time of NEXRAD images. Therefore, NEXRAD im- 
ages should be used in the more strategic sense to avoid areas of convective activity by a 
wide margin. 

Weather information provided by the FISDL display in text form (METAR, TAF, etc.) 
should only be used for gathering an understanding of weather conditions over a large 
geographical area. Other independent sources of information must also be used in con- 
junction with the FISDL display to assure that a complete understanding of the weather 
conditions is obtained. 

Pilots should be fully aware that a weather information display does not contain sufficient 
information to support navigation, and it should not be used as a replacement for any as- 
pect of approved navigation procedures and equipment. While a weather information dis- 
play can increase a pilot’s situational awareness, particularly with respect to weather 
conditions, the display cannot be successfully used to determine headings, direction, or 
distances with the accuracies and reliability that are required for navigation. 

The mental activity required to use a weather information display can increase the pilot’s 
workload in instrument conditions for some pilots. An autopilot can offset this workload 
increase, freeing up the mental processes to support more effective use of the display. 
Some pilots have reported that an autopilot is essential to their effective use of a weather 
infomiation display. 

8.2 Weather Display Manufacturer Recommendations 
8.2.1 Consider Providing Ownship Information 

An overwhelming response from the pilots in this study was the need for ownship posi- 
tion information. Both subjective and objective measures found that most of the pilots 
had difficulty determining their position in relation to the weather and their distance to 
the convective weather activity. With the proliferation of moving map displays in modern 
cockpits, pilots are used to seeing ownship symbology that they use to determine their 
position. 
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The benefits of ownship symbology appear to outweigh the concerns associated with the 
display of real-time position information (ownship ground track) and old information (7 
minute old NEXRAD) on the same display, During the post-flight briefings with the 
subject pilots, many commented that they realized the NEXRAD images were old and 
would take the staleness into account when comparing ownship information in relation to 
the weather depiction. 

8.2.2 Provide Direction and Rate of Hazardous Weather Motion 

Many of the pilots in this study had difficulty determining the movement of the convec- 
tive weather and asked for either a looping capability (playback of preceding images) or 
vector arrows showing speed and direction (similar to the National Weather Service radar 
depiction charts). 

8.2.3 Provide Distance Determination 

Many of the pilots in this experiment made poor estimations of the distance between the 
aircraft and the convective weather. This misperception was a significant contributor to 
the inability of many of the pilots to effectively use the weather information display. This 
indicates the need for a means such as range rings to determine range information on the 
display. 

8.2.4 Provide Intuitive NEXRAD Image Age Information 

There is a concern that the display of stale weather information in the cockpit may cause 
interpretation difficulties and lead to tactical use of stale weather information. The pilots 
that were aware of the staleness of the NEXRAD images used the information correctly, 
several admitted that they just assumed that the information was a consistent age, gener- 
ally about 10 minutes old. This rationale was due to the difficulty and cognitive processes 
required in subtracting the current time from the NEXRAD timestamp. The problem will 
only be exacerbated should some manufacturers choose to offer NEXRAD images on a 
“pay-for-view” basis. 

8.2.5 Provide METAR Code Translation 

In this study, the pilots’ commented that the METAR reports were difficult to interpret, 
took too much time and were not of much use because they were old. Currently the tex- 
tual METAR information is presented in typical teletype codes and although this is the 
infomiation that will be broadcast at no charge to the user, the interpretation of those 
codes in a high workload environment causes many errors. In this study, many errors 
were observed and excessive time was devoted to decoding the METAR information. 

Additionally, what pilots need prior to commencing an approach (via data link) is the cur- 
rent official observation for the airport (METAR or SPECI). Rapidly changing control- 
ling elements such as RVR are best provided (in the near term) directly from the 
TRACON or tower controller who have direct readouts of the current conditions. In the 
future, consideration should be given to the provision (via data link) of direct readouts of 
current controlling conditions to the pilot. 
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8.3 Recommendations for Further Research 

Proposed research topics fall into three broad categories: evaluation of specific issues of 
interest to the FAA, weather information integration and display design enhancements, 
and development of a training system for aircraft cockpit weather information manage- 
ment. 

8.3.1 Conduct Evaluations of Specific Issues to Support Standards 

This research topic is motivated by the interest in current and pending cockpit weather 
displays to be designed and marketed by the manufacturers in the FY01 time frame, and 
the need for standards and design guidelines to assure safety. Issues to be addressed in- 
clude 1) how to minimize the effort required to learn to use the display, 2) how to facili- 
tate interpretation of the display, 3) how to minimize the likelihood that the displayed 
weather information can be misused, leading to poor navigation decisions, and 4) how to 
efficiently obtain and coordinate the data necessary for incorporation in appropriate stan- 
dards. 

8.3.2 Develop Concepts for Integration and Display Enhancements 

A variety of weather related products and onboard sensor derived data are currently 
available or pending implementation in aircraft, e.g., NEXRAD images, lightning data, 
datalinked icing warnings, convective weather turbulence, etc. Display concepts are 
needed which integrate the available data into more useful representations of spatial and 
temporal information incorporating the lessons learned to date by the RTI/NASA team 
and other organizations working in this area. Such integration would reduce the cognitive 
skills required of the pilot to integrate weather data from many sources as well as reduce 
the comprehensive weather data interpretation training currently required. 

Based on the lessons learned from the previous experiments, such display concepts might 
incorporate the following attributes: 1) GPS derived ownship position in real time, 2) 
more intuitive zoom, pan, and ownship centering capabilities and operations, 3) optimum 
NEXRAD cell size, 4) an intuitive indication of NEXRAD data staleness, 5) an intuitive 
indication of map range distances, and 6) borrowing from military applications, depiction 
of a “threat representation” region to indicate those weather related areas for the pilot to 
avoid (precipitation, lightning, icing, turbulence, etc.). 

8.3.3 Develop Training Curriculum for Weather Information Displays 

A training curriculum should be developed to support the implementation and proper use 
of weather displays in the cockpit. The curriculum needs to include appropriate manuals 
and modem interactive multi-media training techniques that would highlight common 
mistakes and improper usage of the weather display information, and develop and rein- 
force appropriate operational procedures for the use of weather display systems. Accom- 
panying experiments should be undertaken to evaluate the efficacy of the training cur- 
riculum. 
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Appendix A. Flight Information Services - Broadcast Description 

System Overview 

Flight Information Services Data Link Display (FIS) will provide pilots with the display 
of certain aeronautical weather and flight operational information. This infomiation will 
be displayed using both text and graphic formats. Service providers will provide a broad- 
cast FIS system using VHF data link. This system will provide coverage throughout the 
Continental United States from 5000 feet AGL to 17,500 feet MSL, except in those areas 
where this is unfeasible due to mountainous terrain. Aircraft equipment will include at 
least an appropriate receiver and display unit. This system will provide, free of charge, 
the following Basic Products: 

Aviation Routine Weather Reports (METARs), 

Special Aviation Reports (SPECIs), 

Temiinal Area Forecasts (TAFs), and their amendments, 

Significant Meteorological Information (SIGMETs), 

Convective SIGMETs, 

Airman’s Meteorological Information (AIRMETs), 

Pilot Reports (both urgent and routine) (PIREPs), and 

Severe Weather Forecast Alerts (AWWs) issued by the FAA or NWS. 

Additional products, called Value Added Products, will be available from the FIS provid- 
ers on a paid subscription basis. Most of the value-added products are expected to be 
graphical in nature and may include but are not limited to: 

National, Regional and Local NEXRAD mosaics 

Icing forecasts 

Turbulence forecasts 

Graphical METARS 

Winds 

Cloud Tops 
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The FIS products will be required to conform to FAA/NWS standards. Specifically, the 
FIS weather information must meet the following criteria: 

1. The products are either FAA/NWS accepted aviation weather products, or 
based on FAA/NWS accepted weather products. 

2. In the case of a product which is the result of the application of a process 
which alters the form, function or content of the base FAA/NWS accepted 
weather product(s), that process must be: 

a) An established, conventional aviation weather process used in stan- 

dard U.S. aviation weather information systems, and, 

b) Managed by a qualified aviation meteorologist. 

National Airspace System (NAS) status products (such as NOTAMs, Special Use Air- 
space Status, etc.) will include verbatim transmissions of FAA products. If graphics are 
used to describe NAS status, the basic text product will be readily available to the pilot 
for reference. 


Operations 

To receive FIS broadcasts, an aircraft must have a data link radio and appropriate display. 
Both of the initial FIS service providers were awarded frequencies between 136.425 MHz 
and 136.500 MHz for broadcast of FIS weather products. The aircraft’s data link radio 
must be tuned to one of the two frequencies to receive weather information from the ap- 
propriate provider. 

Weather information will be broadcast from each ground station at established intervals. 
Upon full deployment, each FIS provider will provide coverage throughout the National 
Airspace System (NAS). 
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Appendix B. Literature Search Results 


Andre, A.D., & Cutler, H.A. (1998). ’’Displaying uncertainty in advanced navigation 
systems." Proceedings of the Human Factors and Ergonomics Society 42 nd Annual 
Meeting. Vol. 1 pp 31-35. 

Summary: 

Uncertainty may involve lack of accuracy, lack of precision, or time lag. 

Position uncertainty. In creating a scenario where the pilot had to avoid collision 
while maintaining relative course, "graphical-implicit" (text #'s, or color) and 
"graphical-explicit" (obstacle surrounded by circle of uncertainty) symbologies aided 
in reducing the number of collosions under conditions of positive uncertainty, relative 
to the no-represtation group. However, only the "graphical-explicit" symbology 
showed benefit under high uncertainty conditions. 

Heading uncertainty - Subjects were found to shoot more bogeys and less friendlies 
when the known display of uncertainty was shown. Graphical depicted "arcs" proved 
most successful in displaying uncertaint headings in enemy and friendly aircraft. Text 
and graphical "rings" also proved beneficial compared to the control condition. 


Aretz, A.J. (1988). “A model of electronic map interpretation.” Proceedings of Hu- 
man Factors Society - 32nd Annual Meeting. 

Summary; 

Map complexity: 

Very powerful effect. Do everything possible to reduce complexity of information con- 
tained on electronic map displays. Any increase of map complexity will increase time to 
use the information. 

Processing: 

To avoid sequential processing, present information simultaneously with HUD or virtual 
display. 
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Banbury, S., Selcon, S., Endsley, M., Gorton, T., & Tatlock, K. (1998) "Being cer- 
tain about uncertainty: How the representation of system reliability affects pilot de- 
cision making." Proceedings of the Human Factors and Ergonomics Society 42nd 
Annual Meeting. Vol. 1 pp 36-39. 

Summary: 

When presented with a single machine-identified target, pilots were unwilling to accept a 
level of uncertainty above 9% (lower than 91% confidence). When a secondary aircraft 
was presented as friendly, pilots were far more reluctant to shoot (when uncertainty level 
was above 3%) or (when confidence level was below 97%). Results suggest that an ex- 
plicit suggestion of the risk of fratricide cause pilots to become more conservative, even 
though they were briefed that all other aircraft were present in equal numbers. 

Participants took significantly longer when two aircraft were presented rather than one. 
When a second plane was presented as a friendly, subjects were far quicker at making the 
shoot/no shoot decision when the uncertainty level was more than 9% (no shoot). 

Reaction times were significantly worse when uncertainty levels were between 6-9%, 
suggesting that participants found it difficult to make a decision faced with information 
bordering on maximum level of risk they would accept for fratricide, which is consistent 
with a study by Selcon (1990) who found that when the probabilities led to some ambi- 
guity as to what to do, decision time was slower than if no probabilities had been pre- 
sented at all. 

There was no difference in whether the information was presented as uncertainty or con- 
fidence. 


Boyer, B., Campbell, M., May, P., Merwin, D., & Wickens, C.D. (1995). "Three- 
dimensional displays for terrain and weather awareness in the national airspace sys- 
tem." Proceedings of Human Factors and Ergonomics Society 39th Annual Meeting. 

pp 6-10. 

Summary : 

Generally no advantage of 3D display over 2D display, except that. 3D display led to tra- 
jectories that were “more conservative,” taking longer paths, that skirted the hazard by a 
wider margin - believed to be related to participants having less certainty regarding lat- 
eral position of the aircraft relative to the hazard, causing them to take a cautious ap- 
proach creating paths far enough away from the hazard to compensate for any possible 
error in their perception of lateral separation. 
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Campbell, M., May, P.A., & Wickens, C.D. (1995) “Perspective displays for air traf- 
fic control: Display of terrain and weather.” Eighth Internationa] Symposium on 
Aviation Psychology. Vol. 1 pp. 375-381. 

Summary: 

Speed advantage for planar displays in categorization of the presence of a threat. 

Top-down view provided a more appropriate domain to make lateral judgements. 

Planar display provided a more accurate means of travelling to the final destination, how- 
ever, the subjects needed to make more vector clearances to do so. 


Cardosi, K., & Hannon, D. (1999). ’’Guidelines for the use of color in ATC dis- 
plays.’’ U.S. Department of Transportation, Federal Aviation Administration. 

Summary: 

Several experts in the field of color vision and displays have compiled recommendations 
on the use of color in electronic aircraft displays. Most recommend a conservative and 
consistent use of color, using no more than six color codes for symbols: white, red, green, 
yellow, magenta, and cyan, while reserving red and yellow for warnings and cautions. 
The use of more than six symbol colors may degrade performance on search, and identi- 
fication, especially under high ambient light. 


Cohen, M.S. "Taking risks and taking advice: The role of experience in airline pilot 
diversion decisions 1.” NASA Contract. 

Summary: 

Through 10 different scenarios, experienced and inexperienced pilots were given paper 
and pencil flight-decision scenarios. Relative to their situation, they were then given a 
worst case, expected case, and best case scenario predictions of the situation which in- 
volved possible combinations of weather and/or fuel problems. Dependent variables were 
decision to divert/continue, and subjects assessment of their confidence of the judgement. 

Subjects fell into 3 general categories of either risk takers, non-risk taking experienced, 
or non-risk taking inexperienced. 

Risk takers were willing to accept the "worse case" scenario of no options if the expected 
case or best case scenario's were good. 

Non-Risk taking experienced pilots were the only ones to take dispatch advice, centering 
their decision making process around the recommendation. 

Non-Risk taking less-experienced pilots fell evenly into two categories: cautious-strategy 
and worst-case strategy. The cautious-strategy saw the worst-case of no options as suffi- 
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cient but unnecessary to cause a diversion, while the worst-case strategy saw the worst- 
case of no options as both necessary and sufficient for diversion. 


Crabill, N.L., & Dash, E.R. (1991) "Pilot's automated weather support system 
(PAWSS) concepts demonstration project - Phase 1 - Pilots weather information re- 
quirements and implications for weather data systems design." FAA Technical 
Center Engineering Field Office, NASA Langley Research Center. 

Summary: 

Flight broken down into stages into pre-flight, take off, departure, climb, cruise, ap- 
proach, landing, and post flight operations. Information needed by pilots at each of these 
stages is listed and diagramed. This information is further broken down into surface 
weather and aloft weather required at each stage. See hard copy for details. 


Dershowitz, A., Lind, A. T., Chandra, D.C. & Bussolari, S.R. (1995). “The effect of 
compression induced distortion of graphical weather on pilot decision making.” 
Eighth International Symposium on Aviation Psychology. Vol. 2 pp. 827-832. 

Summary: 

Subjects were given high resolution and low resolution weather graphics (referred to as 
uncompressed and compression respectively) in order to plan a route to a predetermined 
destination. Results showed that in general, highly compressed data was rated as unac- 
ceptable to the pilots. Also, with highly compressed data, route area error was signifi- 
cantly greater (not necessarily a bad thing- see hard copy), and was associated with 
proximity to weak precipitation. There was no significant differences in route length. 


Driskill, W.E., Weissmuller, J.J., Quebe, J., Hand, D.K., Dittmar, M.J., Metrica, 
Inc., & Hunter, D.R. (1997). "The use of weather information in aeronautical deci- 
sion-making." National Technical Information Service, Springfield, Virginia. 

Summary : 

Investigated the values of worth functions pilots attribute to weather and terrain variables 
in making decisions about flight in a single-engine aircraft under VFR. 

Pilots decision policies were found to vary based on 1) general comfort level; 2) mixture 
of age and general experience 3 ) number of hours flown in last 90 days and 4) reason 
for flying (employed as pilot vs. pleasure flying). 

While compensatory models (one good weather factor compensates for a bad one) for 
go/no go decisions are used, they are not always the best - especially for non-experts. In 
cases where a pilot is inexperienced, compensatory models are sometimes very danger- 
ous, and therefore increased emphasis on risk assessment and self-perception training are 
recommended. 
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Although there was common ordering of weather variable importance (many interactions 
of ceiling, visibility & precipitation), each pilot attributes uniquely different weights to 
weather conditions for each terrain type depending on familiarity with the terrain. 


Fisher, B.D., Brown, P.W., Wunschel, Jr., & Stickle, J.W. (1989) "Cockpit display of 
ground-based weather data during thunderstorm research flights." 27th Aerospace 
Sciences Meeting. 

Summary: 

A prototype system (actual F-106B airplane) was developed to provide a cockpit display 
of ground-based weather data and was used during thunderstorm research. This system 
was severely limited due to small image size and the inability to continuously update the 
data. However, it was found helpful in the selection of the route of flight (strategical de- 
cisions), general track to be used, and occasionally in clarifying the location of a specific 
cell of interest (perhaps tactical). Recommendations for improvement include incorpo- 
rating an airplane heading up display mode with a digital display of heading while re- 
taining the choice of a north-up display mode, also to provide the pilot with control of the 
magnification feature and translation of the displayed area 


Guilkey, J.E., Jensen, R.S., Caberto, S.C., & Fournier, D.L. "Piloting expertise in- 
terven tion strategies for aeronautical decision making." Ohio State University, 
Columbus, Ohio. 

Summary: 

Trends in data indicate that "expert" pilots could be differentiated from "average" or 
"poor" pilots in terms of 1) seeking additional quality information in a more timely man- 
ner, 2) making progressive decisions to solve a problem, and 3) communicating readily 
with available resources. 

Attempt to quantify/qualify characteristics of the "expert" pilot in the realm of problem 
solving, and then to use those methods to teach this problem solving technique to those at 
the lower end of the continuum. 


Hale, S.L., (1988). “Use of color CRT’s in aircraft cockpits: A literature search.” 
U.S. Army Human Engineering Laboratory, Aberdeen Proving Ground, Maryland. 

Summary; 

Luminance: 

Contrast ratio should be between 6:1 to 10:1. 

Bright background best for color visibility. 
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Saturation: 

Highly saturated colors have little benefit and often cause after images. 

Color for coding: 

Red for warning or danger and nothing else should not be used as singular information 
code; use shape or position for redundancy. 

Color is highly useful for high-density displays. 

Number of colors: 

Discrepancies in literature 

Color types: 

Green: recommended as predominant color for coding. 

Yellow: moderate priority. 

Red: high priority - threat or danger. 

Blue: to perceptually separate related or adjacent symbology. Don’t use with shape cod- 
ing, it reduces legibility. 

Desaturated orange: may be used in place of green for sensor imagery or computer im- 
agery. 

Background: 

Dark background provides high contrast, but black too dark. 

Best to have grayish background that remains neutral under ambient illumination. 
Primary display should be of similar brightness as other displays so as not to induce eye 
fatigue. 

Task combinations: 

When choosing colors it is vital to consider all tasks to be performed. 

Cockpit Environment: 

Automatic contrast/brightness adjust system is a must for a cockpit display. 


Hansman, R.J. & Wanke, C. (1989). "Cockpit display of hazardous weather infor- 
mation." 27th Aerospace Sciences Meeting. 

Summary: 

Experiment conducted with a GA simulation to compare voice, text, and graphical depic- 
tion of weather and aircraft position in recognizing and avoiding microbursts. 

Study was extremely weak, using a total of only 8 pilots to evaluate the 3 different con- 
ditions. There is no mention of how these 8 subjects were dispersed among these condi- 
tions, or if they were all used in each condition, resulting in a massive lack of control of 
learning effects. Comparisons are drawn using only percentages, with no other statistical 
data reported. 
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There does exist some anecdotal evidence that verbal relay of microburst alerts lead to 
delays. 


Pilot survey indicated that PIREPS and visual cues are the best currently available meth- 
ods for microburst detection, while LLWAS and airborne weather radar are less effective. 


Hoffman, R.R. (1990). “Human factors psychology in the support of forecasting: 
The design of advanced meteorological workstations.” American Meteorological 
Society. Volume 6. 

Summary : 

Recommendations for design in information processing components of Advanced mete- 
orological presentation, (AMP). 

Design subsystems in modular form. 

Many of the system operations functions, maintenance functions, and hardware aspects 
should be invisible to the user. 

User operation should involve use of icons and menus & minimize control language. 

Promote easy navigation (show “go back”, “escape”, “undo”, and “where am i”) make 
clear the next steps to be taken. 

Formats should be consistent across data types and should contain explanations in text 
form, even if long. 


Hughes, D. (1989) “Glass cockpit study reveals human factors problems.” Aviation 
Week & Space Technology, pgs 32-36. 

Summary: 

Air Transportation Association Study: 

Too much reduction of workload in low workload phases. 

Too much increase of workload (monitoring) in high workload phases. 

Potential for too much “head-down” time. 

Difficulty in recovering from automation failure. 

Reluctance of crews to take over a failing automated system. 

Deterioration of pilot skills. 

Loss of vigilant performance. 

Difficulty in detecting system errors. 

Incompatibility between new aircraft, ATC, and old fleet. 
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Hunter, D.R., Driskill, W.E., Weissmuller, J., Quebe, J., Hand. J., & Dittmar, M. 
(1995). “Analysis of the weights applied to weather information by pilots.” Eighth 
International Symposium on Aviation Psychology. Vol. 2 pp. 833-838. 

Summary : 

Many pilots use a compensatory model for evaluating weather information, however 
there has been no data collected to demonstrate that the use of a compensatory model is 
appropriate for all or even the majority of pilots. It might be argued that in many situa- 
tions (e.g. mountain flying) a non-compensatory model should be used. One such model 
might set a minimum value for ceiling values (e.g. sufficient to clear all mountains) 
which must be met, regardless of the visibility. That is, in such a model, having a very 
high visibility does not compensate for having a low ceiling. 


Kirkpatrick, G.M. (1979). "Real time weather display in the general aviation cock- 
pit." AIAA Aircraft Systems and Technology Meeting. 

Summary : 

Proposed use of VOR voice channel for the relay of current radar charts. The user will 
receive a benefit in completion of additional flights which would previously have been 
delayed or cancelled because of thunderstorms. 


Kochan, J.A. "Aeronautical decision making: the expertise method." The Ohio 
State University Aviation Research Team, Columbus Ohio. 

Summary : 

Based on subjective interviews, a model of attributes of the "expert pilot" was formed. 
Study suggests that expert rating comes from more than just flight hours, but from num- 
ber, variety, meaningfulness, relevancy, and recency. Also suggests risk management, 
attentional control, and dynamic problem solving are key components. See hard copy for 
details. 


Lee, A.T. (1990). "Aircrew decision-making behavior in hazardous weather avoid- 
ance." Aviation, Space, and Environmental Medicine. February, pp 158-161. 

Summary: 

Evaluated the performance of experienced airline crews in the assessment and avoidance 
of microburst events in simulation. Compared conventional ATC transmission of weather 
with 2 display groups receiving visual Doppler returns automatically when the aircraft 
was within a 60 nm radius of the airport. Microburst events occurring within 3 nm of the 
approach or departure ends of the active runway were also displayed. The 2 display 
groups differed only with respect to the time at which the microburst alert was received. 
The control group and display group 1 received the alerts at 3 nm from the airport, while 
display group 2 received alert on the downwind leg of the approach. 
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Real-time visual display of terminal area convective activity exhibited awareness of mi- 
croburst event probability, found from increased communication between crew members 
referencing microbursts. 

Reduction in decision time with visually displayed information averaged nearly 1 min. 


Lind, A.T., Dershowitz, A., & Bussolari, S. (1994). “The influence of data link- 
provided graphical weather on pilot decision-making.” Government Technical Re- 
port No. ATC 215, Lincoln Laboratory, MIT. 

Summary: 

Subject pilots were given prepared flight plans, weather briefings, and graphical weather 
images. This was done in an office setting and graphical images were presented on an 
Apple Macintosh Computer. Subjects were asked to make weather-related decisions 
(both tactical and strategical) without time constraints and the workload demands of ac- 
tual flight. 

Results indicated that all pilots made noteworthy differences in the action taken/decision- 
making. When pilots could see the graphical depiction on GWS, their situational aware- 
ness was better and they were able to make informed GO/NO GO decisions, as well as 
informed in-flight deviations when compared to access to weather information through 
verbal query of ground-personal. 

Pilots made significantly fewer calls when they had GWS. This could reduce workload of 
both pilots, and ATC’s. 

Pilots that had GWS rated their confidence in ability to assess weather higher than those 
without it. 

Pilots with GWS indicated a higher mean hazard rating than pilots who did not have 
GWS for the same flight, although this was not statistically significant. This may allevi- 
ate concerns that with more information the pilots may feel over-confident, and subse- 
quently make poor decisions. 

Pilots reported the GWS system very useful, and worth the estimated cost of $5,000. 


Lindholm, T.A. (1995). “Advanced aviation weather graphics - information content, 
display concepts, functionality, and user needs.” Eighth Internationa] Symposium 
on Aviation Psychology. Vol. 2 pp. 839-844. 

Summary: 

Present only the information needed to accomplish the task, and in a form that requires 
little or no interpretation. 
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Weather data must be presented in the same spatial and temporal context as the task at 
hand. 

Weather displays must be tailored to the individual class of users (based on why the in- 
formation is needed) and how the user will use it. 

Field evaluations must permit open and free user feedback in the operational setting. 

Laboratory evaluations seem to miss essential elements of the task structure, such as 
stress or task saturation. 


May, A. (1997). "Neural network models of human operator performance." The 
Aeronautical Journal. No. 2129 pp 155-158. 

Summary : 

Examines the feasibility of using neural networks to represent the effects of human op- 
erators in computer models of complex man-machine systems. Data from the man-in-the- 
loop simulators are used to train the networks. This method was tested on several data 
sets using a stand alone prototype system with successful results. These results can be 
used to place constraints on the quality, quantity and type of simulator data required in 
future applications. 


Merwin, D.H., O’Brien, J.V., & Wickens, C.D. (1997). “Perspective and coplanar 
representation of air traffic: Implications for conflict and weather avoidance.” 
Ninth International Symposium on Aviation Psychology. Vol 1 pp. 362-367. 

Summary: 

Consistent advantage of the coplanar 2D formats over the 3D perspective format in sup- 
porting traffic avoidance maneuvers. (Ambiguity with which the perspective display de- 
picts position and separation along the line of sight or the viewing axis of the display). 

Clear advantage of data base integration, suggesting that considerable caution should be 
exercised in adding separate monitors or display units for separate hazard data bases. If 
display begins to appear cluttered, then use color or intensity coding or decluttering algo- 
rithims. 

O’Brien, J.V., & Wickens, C.D. (1997). “Free flight cockpit displays of traffic and 
weather: effects of dimensionality and data base integration.” Proceedings of the 
Human Factors and Ergonomics Society 41st Annual Meeting. Vol. 1 pp. 18-22. 

Summary: 

Integrated weather and traffic information tended to result in fewer conflicts than the 
separated displays. 

2D coplanar displays supported better hazard awareness and avoidance than 3D displays. 
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O'Hare D., & Smitheram, T. (1995). " 'Pressing on' into deteriorating conditions: 
An application of behavioral decision theory to pilot decision making." The Interna- 
tional Journal of Aviation Psychology. Vol 5, (4), 351-370. 

Summary : 

Looked at the possibility of changing decision making outcome based on situational 
framing, an approach called "prospect theory." 

Participants were shown to select the gains frame rather than the losses frame as the way 
they would naturally consider the decision of whether to continue with the flight. 

Pilots who viewed the decision from a gains (of turning around) framework were signifi- 
cantly less likely to press on into deteriorating conditions than pilots who viewed the de- 
cision from a loss perspective. 

Pilots should be encouraged to consider in-flight decision about whether to continue with 
a flight in terms of their current position, and should forget about any past losses such as 
money spent, time and fuel wasted, and pilots should also be encouraged to make in- 
flight decisions in terms of gains rather than losses, making them more likely to make 
risk-averse choices. 


Rahman, T., & Muter, P. (1999). "Designing an interface to optimize reading with 
small display windows." Human Factors, Vol. 41, No. 1. UOT, Toronto, Ontario. 

Summary : 

Experiment 1 concluded that efficiency was poorer in the Rapid Serial Visual Presenta- 
tion (RSVP- single word after single word) than other conditions. Efficiency was at least 
as high in the sentence to sentence condition as it was in a normal paragraph condition. 
Experimenters introduced a visual completion meter below the word/words to let the 
reader know how much further the sentence or paragraph had until its end. This did not 
detract from performance, and in fact significantly improved sentence by sentence com- 
prehension, and was preferred by readers. 

Faster presentation of RSVP in Experiment 2 perhaps eliminated the poor performance. 
RSVP was shown to be just as efficient as sentence by sentence, and normal paragraph 
form, although it was the least preferred. 

Most other experiments cited, found that the "Times Square" method (text scrolling 
across the screen from right to left) resulted in inferior performance compared with 
RSVP and page format. See hard copy for examples of text. 
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Rehmann, A. J. (1995). “A Pilot Evaluation of Text Display Formats for Weather 
Information in the Cockpit.” FAA Technical Center, Atlantic City, N.J., 
DOT/FAA/CT-TN95/42. 

Summary: 

Pilots were given various text displays of in-flight weather reports in a part task simula- 
tion exercise. The weather reports were presented in both teletype code formats and plain 
English formats with either a vertical or horizontal orientation. Additionally, various data 
entry techniques were explored, such as: keyboard entry, cursor select, bezel key select 
and number selection. Both general aviation and airline pilots were used as evaluators. 

Results indicated that the vertical format of text presentation was preferred for its read- 
ability, organization and error reduction. The general aviation pilots had difficulty inter- 
preting the teletype code formats because of their unfamiliarity of using those codes in a 
day to day environment, as the airline pilots are used to. Therefore, the general aviation 
pilots showed better accommodation, and reduced errors, to the plain English format. 

The pilots were asked to enter the station identifier using various data entry methods. The 
results showed that the bezel key select method was superior from both an operational 
view and error reduction view. 


Rothenheber, E., Stokes, J., LaGrossa, C., Arnold, W., & Dick, A.O. (1990). 
"Cockpit Ocular Recording System (CORS)." Prepared for Langley Research Center 
under contract. 

Summary: 

Experimental Issues: 

Findings of experiments suggest the CORS oculometer accuracy is variable and is de- 
pendent on, at least, the following: 1) time since calibration 2) location in the visual 
field and 3) test subject, including the size of pupil and other physiological variables. 


Tullis, T.S. (1988). "Screen Design" -{Chapter 18 of Handbook of Human-Computer 
Interaction M. Helander (ed.)). Elsevier Science Publishers 

Summary: 

Recommendations on screen design: 

Most important: Present only that information that the user needs, no more or less. Dis- 
play most used items on primary display, use a function key or something to access in- 
frequently used items. Presenting them simultaneously with often used items only brings 
clutter. 

Abbreviations: Only to be used when they are standard and familiar to all. 
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Use familiar forms: For example a persons name, street address, city, state, and zip code 
are a familiar format, and therefore don't necessarily need to be labeled - therefore saving 
space. 

Use tabular column headings: This will save you from labeling something time & again. 

Make use of grouping: Grouping similar items together in a display format improves their 
readibility and can highlight relationships between different groups of data. 

No more than 5 degrees: 5 degrees of visual angle to which the eye is the most sensative 
(approximately the foveal region of the retina) shows data in chunks that can be taken in 
at one fixation, and will help speed search time. This visual angle, assuming average dis- 
play characteristics and viewing distance, translates to an area about 12-14 characters 
wide and 6-7 lines high. 

Better to have few groups defining many variables, than may groups defining a few vari- 
ables. 

Spacing is better than use of color for separating or grouping. 

If using brightness as a distinguisher of information, only use two different levels - any 
more than that is difficult to discriminate. 

Flashing should only be used as imminent danger, if used at all. 

Search times are shortest for top left, and longest for bottom right. 

General elements should precede more specific ones. 

Numbers should be tight justified or decimal point aligned. 

Use indentation to represent hierarchical relationships. 

Usually more effective to put a label to left of item, except when using tabular column 
headings. Also should use dotted line connectors . 

Writing words in upper and lower case will increase reading speed by about 13%. 

When looking to draw attention, use all caps. All caps, is read faster than all small. 

Text with consistent spacing should always be used, despite ragged right margins. 

Space between bottom of one line, and the top of another should be about equal to or 
slightly larger than a letter. 

Space used between paragraphs helps to group concepts. 
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Line length should not exceed more than about 60 characters, unless line spacing is in- 
creased . 


Wiener & Curry (1980). "Automation guidelines: Appendix 1." pp 189-190. 
Summary : 

System operation should be easily interpretable and understandable. 

Automation must perform how the operator wants it to perform, not at some lower stan- 
dard, otherwise it won't be used by the operator and it would have cost you time and 
money. 

Design automation to prevent peak levels of task demand from becoming excessive, en- 
suring available time for monitoring. 

Allow for different operator styles when feasible - (choice of automation). 

Make sure system performance will be insensitive to different modes or options (i.e. the 
pilot may choose to have the autopilot either fly pilot-selected headings or track ground- 
based navigation stations). 

Provide means for checking the set-up and information input to automatic systems. 

Extensive training required to ensure proper operation, and to implement correction pro- 
cedures. 

Provide meaningful duties when automation reduces task demand to low levels. 

Keep false alarms at acceptable rates. 

Alarms with more than one mode or responsible for more than one condition must indi- 
cate which condition caused the alarm. 

Provide a quick way to check the validity of alarms. 

Format of alarm should indicate degree of emergency. 
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Wickens, C.D., & Scott, B. (1983). "A comparison of verbal and graphical informa- 
tion presentation in a complex information integration decision task." Office of Na- 
val Research Engineering Psychology Program. 

Summary 

Experiment looked at verbal vs. spatial-graphical display formats in presenting sequential 
information for a tactical decision making task. Subjects' task involved integrating infor- 
mation to determine which one of two tactical battlefield maneuvers was in effect. Sub- 
jects were more accurate using the spatial display which supports the stimulus-central 
processing compatibility theory stating that the analog operations on which the judge- 
ments were based would be best served by spatial displays. This graphical advantage was 
enhanced when cues in both subject groups were delivered at a slower speed, imposing 
greater demands on working memory in the verbal group. 


Wiggins, M., Connan, N., & Morris, C. “Self-perceptions of weather-related deci- 
sion-making Ability amongst pilots.” University of Newcastle, Newcastle, Austrailia 

Summary: 

Results suggest that inexperienced pilots are more inclined to rely upon their self- 
perceived risk-taking behavior than their self-perceived ability to resolve various deci- 
sions. More experienced pilots’ performance, during complex tasks, is related primarily 
to their perceptions of their own ability to cope efficiently with a situation. 


Wiggins, M., Connan, N., & Morris, C. "Weather-related decision making and self- 
perception amongst pilots." Applied Aviation Psychology. 

Summary: 

Support for perception that risk-taking propensity is indicative of pilot performance dur- 
ing simulated weather-related decision making scenarios. More specifically, a negative 
relationship was found between self-perception of risk and the frequency with which in- 
formation screens were accessed during the scenario. 

Suggest that training must focus on the relationship between risk-taking behavior and the 
process of decision-making rather than simply the outcome of risk taking behavior. 


Wiggins, M., & O'Hare, D. (1995). "Expertise in aeronautical weather-related deci- 
sion making: A cross-sectional analysis of general aviation pilots." Journal of Ex- 
perimental Psychology: Applied. Vol. 1, No. 4 pp 305-320. 

Summary : 

There was qualified support for the notion that through task-specific experience, indi- 
viduals develop procedures that can be generalized and applied subsequently to a variety 
of situations, however the performance between groups was delineated more effectively 


79 



by the specific type of experience being measured, than mere flying hours alone, because 
task related experience requires participation of that particular task. 


Inexperienced pilots accessed a greater number of information screens, made a greater 
number of information recursions, and spent more time examining the information 
screens than the experts. 

Inexperienced pilots also exhibited a greater response latency in selecting whether to 
continue the flight or turn around - base on the weather scenario 

Significantly more intermediate and expert pilots chose to continue the flight (well dvised 
in virtually all cases) than did the inexperienced pilots. 


Williams, A.J., & Harris, R.L. (1985). "Factors affecting dwell times on digital dis- 
playing." NASA Langley Research Center, Hampton, VA. 

Summary : 

In an experimental flight simulator, round dial meters demonstrated shorter dwell times 
and fewer dwells per meter change than the digital displays. The following factors af- 
fected digital display scanning behavior: 1) number of digits 2) update rate of the digits 
3) display media, and 4) character font (The digit size used here (.28-. 50 inches) did not 
affect scan behavior measures). 
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Appendix C. The Risk Assessment Task (RAT) 


Index Of Risk Taking Predilection 

In the subject experiment, pilots were allocated to experimental groups in part on the ba- 
sis of their scores on a risk assessment test (RAT). This appendix briefly describes the 
history, pedigree, and rationale for employing the risk assessment test. 

Rationale 

Behaviors such as the dangerous misuse of weather displays need only take place once 
every few hundred hours of flight to have a significant adverse impact on flight safety 
and the incident/accident statistics. However, such a behavior is unlikely to be spotted in 
an hour or two of flight simulation, even if other realistic features of the operational envi- 
ronment are faithfully reproduced. Purely random sampling of the pilot population, 
therefore, may not expose potentially unsafe behavior. 

The challenge, therefore, has been to utilize expert knowledge of flying, accident causa- 
tion and aviation psychology to recreate the kind of environment and circumstances that 
could be expected to increase the probability of detecting misuse of the new weather dis- 
play technologies. The experiment is not designed to calculate the prevalence of these 
“misuse behaviors”, but rather to evaluate and assess whether and how such misuse could 
occur, thus providing guidance for pilots and display manufacturers. With this in mind 
the study team elected to proceed with a stratified random sample of pilots taken from 
populations that might be identified as higher risk and lower risk pilots. If weather dis- 
play misuse accidents are going to occur, then pilots low in weather knowledge sophisti- 
cation, high in risk acceptance, and motivated to continue a flight seem likely to be over- 
represented in the incident/accident statistics. Therefore, the subjects were pre-screened 
for weather knowledge and risk aversiveness. The purpose of the RAT test is to increase 
the probability of including subject pilots who might exhibit behaviors that would other- 
wise only emerge in the operational environment. The RAT task is not advertised as a 
definitive biographical variable nor as a definitive measure or predictor of pilots’ decision 
making prowess. The RAT does, however, have a well-documented history as a psycho- 
metric instrument, and, indeed, in a range of applied psychological studies including 
aviation, as outlined below. 

Given the risk construct used in the test, the research evidence to date, and the absence of 
alternative screening methods for our purpose, use of the RAT is a rational and low-risk 
option. The subject study is not designed to address questions relating to the relationship 
between risk scores and decisions made in simulated flight. The study seeks to reproduce 
and characterize misuses of the weather display in an operational environment. The ab- 
sence of significant relationships observed between risk, as measured by the RAT, and 
other elements in the study, will not adversely impact the experimental outcome. 
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Origins of the RAT 

The RAT task is one sub-task from a version of a multiple task computerized battery of 
cognitive tasks that was explicitly designed to evaluate aviators 1 (see, for example, “Neu- 
ropsychological screening of aviators: A review,:” Banich, Stokes & Elledge, 1989). The 
original research began with an information-processing task analysis of aviation and ini- 
tially identified six primary areas of aviator cognitive proficiency that the battery should 
cover: working memory, attention (divided and focussed), spatial ability, logical reason- 
ing, perceptual-motor abilities, and processing flexibility (or prioritizing). Risk taking 
predilection or aversiveness, that is, risk judgment, was subsequently added, as this was 
clearly a source of pilot variance not captured under the original six headings. 

Test sensitivity testing, reporting, and reliability 

All of the subtasks were tested against each other empirically in a series of discriminant 
analyses (reported in, for example, “Testing the tests - An empirical evaluation of 
screening tests for the detection of cognitive impairment in aviators.” Stokes 2 , et al. 1991, 
a.). The sensitivity, specificity and positive predictive value for each subtask in all seven 
areas of cognitive proficiency were compared, and thus an objective basis for comparison 
in standard epidemiological terms was determined. Stokes 3 (1999) showed that the tests 
in the battery are reliable, don’t suffer from undue practice effects, and factor load on the 
appropriate constructs. The risk task is particularly strong in these respects, exhibiting no 
practice effect and, whereas certain tasks (e.g. maze tracing, hidden figures recognition 
and spatial memory) all factor load onto one construct (e.g. spatial ability), only the RAT 
factor loaded on the risk construct. 

Applications of the RAT 

The original and updated versions of the battery have been utilized extensively in a range 
of applied studies, (e.g. effects of alcohol on performance 4 , of the artificial sweetener as- 
partame 5 (NutraSweet); of stress and trait-anxiety 6 , novice versus expert performance 7 
and so on). Several studies have resulted in findings that involve the risk construct. 

For example, risk assessment appears to be a specific ability or cognitive dimension 
which can be directly impaired by neurological deficit. In a clinical study, Stokes 2 et al 
(1991, b.) showed that pilots evidenced less propensity for risk taking than members of a 
group of subjects with known neuropsychological diagnoses (Testing the Tests, p.785). 
Moreover, the range of conditions in these diagnoses was broad, including as it did cere- 
bro-vascular conditions, trauma, neurodegenerative disease and sequelae of alcoholism. 

An effect often observed and commented on in the engineering psychology literature 
(see, for example, Wickens, 1984), is the apparent conservatism associated with age. The 
clinical study also scrutinized the extent to which older subjects become more risk 
averse. Generally speaking, risk taking did indeed decrease with age in pilots, while it 
increased with age in the clinical group. The mechanism underlying the latter, clinical 
finding is not well understood, but the RAT findings for pilots are consistent with the 
wider literature on risk and risk aversiveness, increasing confidence in the utility of the 
measure. 
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The RAT task has also been used in a series of double-blind clinical studies of the cogni- 
tive effects of aspartame (in the artificial sweetener ‘NutraSweet’) and of alcohol upon 
pilots. Aspartame was not found to affect cognition in either acute or chronic dosing, but 
alcohol, as anticipated, did. However a number of new effects of alcohol were identified, 
including an increased variability in risk taking. Mean RAT scores were the same in the 
alcohol and non alcohol conditions, but this conceals a significantly greater variance 
around the mean in the alcohol condition. Scrutiny of the data showed that this was not a 
group level effect, but indeed did arise from greater capriciousness in trial to trial re- 
sponses of individuals as they worked through the RAT. Reproduced in the operational 
environment such swings from conservative, risk averse responding to high risk gambling 
could be expected to have a negative impact on safety. 

The RAT and Flight Training 

In a 1995 study, the effect upon flight training success of a number of information proc- 
essing variables, including risk predilection, was examined in the context of university 

o 

flight training (Stokes & Bohan, 1995). This study also evaluated the predictive utility of 
anxiety scores and academic grades. A major influence upon the outcomes of such stud- 
ies is the nature of the criterion of success. In the 1995 study, several criteria were ex- 
amined, including a checkride score, hours to solo, landings to solo, and ground school 
grade. The first three of these are closely associated with psychomotor skill, as the crite- 
ria involve maneuvering flight, rather than primarily cognitive skills such as those in- 
volved in cross-country flight management. The criteria used are reflected in the results - 
dual-task tracking tasks best predict success where maneuvering flight is the criterion. 

An unanticipated finding was that the risk task was predictive of ground school perform- 
ance (which presumably includes a more “cerebral” element and little psychomotor con- 
trol). The effect, however, was weak. In this study important additional criteria were ex- 
amined. Results were compiled for checkride “passers” and “failers”, as one might ex- 
pect, but also for individuals who had not been permitted to take the checkride. 

The significance of this may not be immediately obvious. It is necessary to know that 
instructors were required to “sign off’ a student as being ready for the checkride. 
Moreover, the sign off required that the student fly solo prior to the checkride. Under- 
standably instructors do not wish to be the agent of someone’s demise, and will not per- 
mit those at risk to fly solo. They are, therefore, dropped from the flightcheck pool too. 

Given this, it can be argued that the real dichotomy is not between checkride “passers” 
and “failers”. A bigger performance gap presumably exists between those students signed 
off and those not signed off for the checkride, than between persons passing and failing 
the ride (all of whom had been adjudged fit to fly unaccompanied). In this light the RAT 
scores were revisited. In fact, the highest risk scores were seen among the “not recom- 
mended” group (significantly higher than “failers”). “Passers” and “failers” did not differ 
significantly on the risk dimension. A compelling explanation for these results (and one 
supported by instructor comment) is that during flight training instructors had observed, 
among other defects such as poor psychomotor control, unsafe (“risky”) behaviors in 
certain students and had declined to sign them off for solo or for the checkride. Although 
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unknown to the instructors, these students indeed did have elevated risk test scores in the 
battery administered three months earlier, before the student had commenced flight 
training at all. 
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Appendix D. Weather Knowledge Questionnaire and Key 


The Weather knowledge test presented here is the key used to grade the test. All graded 
answers are shown in grey highlight. All other questions were not graded as they were 
not relevant to weather knowledge. 
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General Aviation Questionnaire 


Thank you for participating in our Research Triangle Institute/NAS A/FA A evaluation of 
advanced aviation technologies. We would like to learn a little more about your aviation 
knowledge before you participate in our study. Please take a couple of minutes to answer 
a few questions. Your answers are strictly confidential and will not be released. 

Name: Date: 

Phone number: E-Mail: 

1. How many years have you been a pilot? 

2. What is your level of pilot certification (circle one)? 

Recreational Private Commercial Airline Transport 

3. What is your approximate number of total flight hours? 

4. Are you an instrument rated pilot? 

If so, are you current to fly instruments? 

5. What does a narrow temperature/dewpoint spread mean? 

Possible Fog 

6. How many feet are there in a statute mile? 

7. What does RYR stand for, and what does it mean? 

Runway Visual Range. Visibility down Specific Runway 

8. What COMM frequency can you use to contact Flight Watch? 122,2, 122.0 

9. Briefly describe class C and class G airspace. 

Class C: 

Class G: 
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10. How much does 20 gallons of 100 LL fuel weigh? 

11. What instrument indications would you notice, on take-off, if the static ports were 
blocked? 


12. What are the altitude limits of class A airspace, and what flight rules apply when fly- 
ing in that airspace? 


13. If you are flying eastbound, and you have a tailwind, would you typically be north or 
south of a low pressure zone? 

South 


14. On a surface analysis weather chart, what do closely spaced isobars mean? 

High Winds 

15. What are you likely to see on the instruments if a pitot tube becomes blocked during 
the enroute phase of flight? Describe each phase. 

Level (accelerating): 

Climb: 

Descent: 


16. In what weather products can you find icing information? 

PIREPs, SIGMETs, TAFs, AIRMETs, Area Forecasts. Prognosis Charts. Com- 
posite Moisture Charts. Wind Aloft Tables 

17. What do boundary layer air, and surface winds near the ground have in common? 

Both are slower than surrounding air, due to friction of the surface. 
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18. On a weather chart, what do the following symbols stand for? 



A. 


Occluded Front 



Stationary Front 


19. What type of information is found in an FDC NOTAM? 

Regulatory Notices. Charting Changes 

20. If a thunderstorm is identified as being severe, or giving an intense radar echo, what 
does the AIM say about how far you should avoid the storm? 

20 miles 

21. What do the following METAR/TAF weather codes stand for? 


RA = _ 

Rain 

SQ = _ 

Squall 

BR = _ 

Mist 

FZ = _ 

Freezing 

FC = _ 

Tornado 

DZ = 

Drizzle 

SH = _ 

Showers 

FU = _ 

Smoke 

FG = _ 

Fog 

GR = _ 

Hail 

SN = _ 

Snow 

IC =_ 

Ice Crystals 

HZ = 

Haze 

TS = 

Thunderstorm 


22. What is a void time clearance? 
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23. On a radar summary chart, what does the notation “NA” mean? 


Not available 


24. During a night-time IFR flight, what clues suggest airframe icing? 


25. Please translate the following METAR weather report: 

METAR KDCA 291554Z 26012G18KT 10SM SCT040 BKN100 

15/05 A2985 


Regan National, 29 th day, 1554 Zulu time, wind 260 degrees at 12 knots - 
gusting to 18 knots, visibility 10 statute miles, scattered clouds at 4000 feet, 
broken clouds at 10,000 feet, temperature 15 degrees Celsius, dewpoint 5 de- 
grees Celsius, altimeter 29.85 inches of Mercury. 


Thank you for taking the time to complete our questionnaire, we appreciate your help. If 
we select you for our simulator study of advanced technologies, we will contact you by 
phone or E-mail. 
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Appendix E. Subject Pilot Screening Test Results and Statistics 


Subject Pilot Screening Test Results and Statistics 


Total Subjects: 




57 



Mean Risk Score: 




3.79 



Risk Standard Deviation: 



0.75 



Mean Wx Knowledge Score: 



27.16 (out of 39 possible) 

Wx Knowledge Standard Deviation 

■ 


6.20 



Subject 

Raw Risk 

Converted 

Raw Wx 

Risk 

Wx Know 

Half Sum 

Group 

Number 

Score 

Risk Score 

Score 

z-score 

z-score 

z-score 

Selection 

1 

4.05 

5.95 

28.00 

-0.35 

0.14 

-0.11 


2 

2.90 

7.10 

22.00 

1.18 

-0.83 

0.18 


3 

3.80 

6.20 

38.00 

-0.02 

1.75 

0.87 

Treatment #1 

4 

3.75 

6.25 

30.00 

0.05 

0.46 

0.25 


5 

3.60 

6.40 

17.00 

0.25 

-1.64 

-0.69 

Control #2 

6 

2.65 

7.35 

35.00 

1.52 

1.26 

1.39 

Treatment #1 

7 

2.40 

7.60 

30.00 

1.85 

0.46 

1.16 

Treatment #1 

8 

3.20 

6.80 

33.00 

0.78 

0.94 

0.86 

Control #1 

9 

4.20 

5.80 

26.00 

-0.55 

-0.19 

-0.37 


10 

3.45 

6.55 

27.00 

0.45 

-0.03 

0.21 


11 

3.55 

6.45 

33.00 

0.32 

0.94 

0.63 

Alternate #1 

12 

3.00 

7.00 

24.00 

1.05 

-0.51 

0.27 


13 

3.95 

6.05 

32.00 

-0.22 

0.78 

0.28 


14 

4.20 

5.80 

30.00 

-0.55 

0.46 

-0.05 


15 

3.65 

6.35 

21.00 

0.18 

-0.99 

-0.40 


16 

3.50 

6.50 

30.00 

0.38 

0.46 

0.42 


17 

3.40 

6.60 

31.00 

0.52 

0.62 

0.57 

Alternate #1 

18 

4.30 

5.70 

21.00 

-0.68 

-0.99 

-0.84 

Control #2 

19 

2.85 

7.15 

22.00 

1.25 

-0.83 

0.21 


20 

5.05 

4.95 

12.00 

-1.68 

-2.44 

-2.06 

T reatment #2 

21 

3.50 

6.50 

30.00 

0.38 

0.46 

0.42 


22 

4.00 

6.00 

28.00 

-0.28 

0.14 

-0.07 


23 

4.00 

6.00 

23.00 

-0.28 

-0.67 

-0.48 

T reatment #2 

24 

4.60 

5.40 

26.00 

-1.08 

-0.19 

-0.64 

Control #2 

25 

3.80 

6.20 

30.00 

-0.02 

0.46 

0.22 


26 

4.55 

5.45 

20.00 

-1.02 

-1.15 

-1.09 

"no-show" 

27 

4.60 

5.40 

28.00 

-1.08 

0.14 

-0.47 

Alternate #2 

28 

3.55 

6.45 

32.00 

0.32 

0.78 

0.55 

Alternate #1 

29 

4.90 

5.10 

34.00 

-1.48 

1.10 

-0.19 


30 

2.65 

7.35 

28.00 

1.52 

0.14 

0.83 

Treatment #1 
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Subject Pilot Screening Test Results and Statistics (Concluded) 


Subject 

Raw Risk 

Converted 

Raw Wx 

Risk 

Wx Know 

Half Sum 

Group 

Number 

Score 

Risk Score 

Score 

z-score 

z-score 

z-score 

Selection 

31 

3.60 

6.40 

32.00 

0.25 

0.78 

0.52 


32 

3.60 

6.40 

24.00 

0.25 

-0.51 

-0.13 


33 

2.60 

7.40 

27.00 

1.59 

-0.03 

0.78 

Control #1 

34 

3.50 

6.50 

26.00 

0.38 

-0.19 

0.10 


35 

3.70 

6.30 

28.00 

0.12 

0.14 

0.13 


36 

4.35 

5.65 

19.00 

-0.75 

-1.32 

-1.03 

Control #2 

37 

3.95 

6.05 

26.00 

-0.22 

-0.19 

-0.20 


38 

4.25 

5.75 

23.00 

-0.62 

-0.67 

-0.64 

Treatment #2 

39 

4.75 

5.25 

30.00 

-1.28 

0.46 

-0.41 

Alternate #2 

40 

3.00 

7.00 

18.00 

1.05 

-1.48 

-0.21 


41 

5.20 

4.80 

37.00 

-1.89 

1.59 

-0.15 


42 

3.60 

6.40 

19.00 

0.25 

-1.32 

-0.53 

Treatment #2 

43 

2.65 

7.35 

30.00 

1.52 

0.46 

0.99 

Control #1 

44 

5.75 

4.25 

20.00 

-2.62 

-1.15 

-1.89 

Control #2 

45 

3.30 

6.70 

36.00 

0.65 

1.43 

1.04 

Treatment #1 

46 

4.35 

5.65 

30.00 

-0.75 

0.46 

-0.15 


47 

4.05 

5.95 

38.00 

-0.35 

1.75 

0.70 

Control #1 

48 

3.05 

6.95 

29.00 

0.98 

0.30 

0.64 

Alternate #1 

49 

4.95 

5.05 

16.00 

-1.55 

-1.80 

-1.68 

Treatment #2 

50 

5.25 

4.75 

24.00 

-1.95 

-0.51 

-1.23 

Treatment #2 

51 

4.20 

5.80 

21.00 

-0.55 

-0.99 

-0.77 

"no-show" 

52 

4.15 

5.85 

32.00 

-0.48 

0.78 

0.15 


53 

3.45 

6.55 

38.00 

0.45 

1.75 

1.10 

Control #1 

54 

3.95 

6.05 

13.00 

-0.22 

-2.28 

-1.25 

Control #2 

55 

2.45 

7.55 

32.00 

1.79 

0.78 

1.28 

Control #1 

56 

3.35 

6.65 

27.00 

0.58 

-0.03 

0.28 


57 

3.30 

6.70 

32.00 

0.65 

0.78 

0.72 

Treatment#! 
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Appendix F. Pre-Flight Weather Briefing 


Pre-Flight Weather Briefing 

As part of the mission preflight briefing materials, each pilot was given a paper copy of a 
standard weather briefing that would have been received by a call to a Flight Service Sta- 
tion telephone briefer. Both the teletype coded reports were given as well as an english 
translation. 

Standard Pre-Flight Weather Briefing 
Adverse Conditions: 

AIRMET (WA) TANGO FOR TURB VALID UNTIL 2721 00Z 

AIRMET TURB...MD VA NC 

FROM EMI TO SBY TO RDU TO PSK TO EMI 

AFT 18Z OCNL MOD TURB BLW 060 DUE TO INCRG SWLY FLOW AHD OF 
CDFNT. CONDS SPRDG EWD AND CONTG BYD 21Z THRU 03Z. 

AIRMET (WA) TANGO for turbulence valid until twenty-one hundred universal coordi- 
nated time for Maryland, Virginia, and North Carolina. 

From Westminster (EMI), Virginia to Salisbury (SBY), Maryland, to Raleigh-Durham 
(RDU), North Carolina to Pulaski (PSK), Virginia to Westminster (EMI), Virginia. 

After one, eight, zero, zero, universal coordinated time, occasional moderate turbulence 
below six thousand feet due to increasing southwesterly flow ahead of cold front. Condi- 
tions spreading eastward and continuing beyond twenty-one hundred universal coordi- 
nated time, and through zero, three, zero, zero universal coordinated time. 

Synopsis: 

At one, seven, zero, zero universal coordinate time, a Cold Front extending from south- 
west Pennsylvania along the Appalachians through Central West Virginia, Western Vir- 
ginia, and Eastern Tennessee, northwest Georgia and Central Alabama will continue to 
move Eastward. 

A Warm Front extending from southwest Pennsylvania Eastward to Atlantic City, NJ. 
will continue to move Northeastward, and a Trough of Low Pressure extending from 
northwest West Virginia southward into Central South Carolina will continue moving 
Eastward. 

Current Conditions: 

PHF SA 1800Z M 8 BKN 07 58/53/0910/992 

Newport News, Williamsburg International Airport weather report, one, eight, zero, zero 
universal coordinated time. Measured ceiling eight hundred broken, visibility seven, tem- 
perature five eight, dew point five, three, wind zero niner, zero at ten, altimeter two, 
niner, niner, two. 
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RIC SA 1800Z 50 SCT M70 BKN 05 57/53/3010/992 

Richmond International Airport weather report, one, eight, zero, zero universal coordi- 
nated time. Five thousand scattered, measured ceiling seven hundred broken, visibility 5, 
temperature five, seven, dew point five, three, wind three, zero, zero at one zero, altime- 
ter two, niner, niner, two. 

OFP SA 1747Z E50 BKN 150 OVC 10 65/53/2015/960 

Richmond, Hanover County Airport weather report, one, seven, four, seven universal co- 
ordinated time. Estimated ceiling five thousand broken, one, five thousand overcast, visi- 
bility one, zero, temperature six, five, dew point five, three, wind two, zero, zero, at one, 
five, altimeter two, niner, six, zero. 

LKU SA 1750Z 20 SCT E40 BKN 100 OVC 10 63/53/2415G20/955 

Louisa County, Freeman Airport weather report, one, seven, five, zero universal coordi- 
nated time. Two thousand scattered, estimated ceiling four thousand broken, one, zero 
thousand overcast, visibility one, zero, temperature six, three, dew point five, three, wind 
two, four, zero at one, five gusting two, zero, altimeter two, niner, five, five. 

WAL SA 1749Z CLR BLO 120 10 61/50/1810/969 

NASA, Wallops Airport weather report, one, seven, four, niner universal coordinated 
time. Clear of clouds below one, two thousand, visibility one, zero, temperature six, one, 
dew point five, zero, wind one, eight, zero at one, zero, altimeter two, niner, six, niner. 

MFV SA 1753Z CLR BLO 120 10 64/52/1806/968 

Accomack County Airport, Virginia weather report one, seven, five, three universal co- 
ordinated time. Clear of clouds below one, two thousand, visibility one, zero, temperature 
six, four, dew point five, two, wind one, eight, zero at six, altimeter two, niner, six, eight. 

SBY SA 1750Z CLR BLO 120 10 60/50/1810/969 

Salisbury, Maryland weather report one, seven, five, zero universal coordinated time. 
Clear of clouds below one, two thousand, visibility one, zero, temperature six, zero, dew 
point five, zero, wind one, eight zero at one, zero, altimeter two, niner, six, niner. 

UA: /OV RIC150015 /TM 1720Z /FL 040/TP Cl 80 /SK SCT150 /TB LGT 

Pilot report one-five miles southeast of Richmond, Virginia. At one, seven, two, zero 
universal coordinated time. At four thousand feet, a Cessna one, eighty reported in clouds 
with light turbulence. 

UA: /OV SBY /TM 1715Z /FL 030 /TP MO20 /SK SCT150 /TB NEG 

Pilot report over Salisbury, Maryland at one, seven, one, five universal coordinated time. 
At three thousand feet, a Mooney reported clouds at one five thousand scattered, and 
negative turbulence. 
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UA: /OV RIC045025 /TM 1710Z /FL 040 /TP C172 /TB LGT-MOD 

Pilot report two-five miles northeast of Richmond, Virginia at one, seven, one, zero uni- 
versal coordinated time. At four thousand feet, a Cessna one, seven, two reported light to 
moderate turbulence. 

Satellite Imagery indicates several Cumulus clouds beginning to develop throughout 
central Virginia, including the Richmond area, over the past hour. 

Weather Radar at one, seven, one, zero universal coordinated time indicates scattered ar- 
eas of light to moderate rain showers in Central Virginia, but no precipitation in the East- 
ern sections of the state. 

En-Route Forecast: 

TAF KPHF 271729Z 271818 16014G24KT P6SM SCT100 BKN200 BECMG 2022 
16017G27KT SCT060 OVC120 

FM0000 1618G25KT P6SM SCT030 OVC060 TEMPO 5SM -SHRA OVC030 
PROB40 0103 VRB20G40KT 2SM TSRA QVC020 CB 

Terminal area forecast for Newport News-Williamsburg International Airport. After one, 
eight zero, zero universal coordinated time, wind one, six, zero at one, four gusting two, 
four, visibility unrestricted, scattered clouds at one, zero thousand, broken clouds at two, 
zero thousand. Conditions becoming between two, zero, zero universal coordinated time 
and two, two, zero, zero universal coordinated time, wind one, six, zero at one, seven 
gusting two, seven, scattered clouds at six thousand, overcast at one, two thousand until 
zero, zero, zero, zero universal coordinated time. 

Central and Eastern Virginia Area Forecast: 

271800Z SCT-BKN050 OVC120 TOP 200, OTLK VFR TSRA 

The area forecast for Central and Eastern Virginia after one, eight, zero, zero universal 
coordinated time: scattered to broken clouds at five thousand, overcast at one, two thou- 
sand, tops at two, zero thousand, outlook VFR with thunderstorms and rain. 

TAF KRIC 271729Z 271818 18018G20KT P6SM SCT060 OVC120 BCMG2022 
OCNL -SHRA QVC030 PRQB40 2302 VRB20G40KT 2SM TSRA QVC020CB 

Terminal area forecast for Richmond International Airport. After one, eight, zero, zero 
universal coordinated time, wind one, eight, zero at one, eight gusting two, zero, visibility 
unrestricted, scattered clouds at six thousand, overcast at one, two thousand. Conditions 
becoming between two, zero, zero, zero universal coordinated time and two, two, zero, 
zero universal coordinated time, occasional light rain showers, overcast at three thousand, 
with a chance of thunderstorms after two, three, zero, zero universal coordinated time. 

TAF KSBY 271729Z 1818 1820G30KT SCT100 BKN200 

Terminal area forecast for Salisbury, Maryland after one, eight, zero, zero universal coor- 
dinated time. Wind one, eight, zero at two, zero gusting three, zero, scattered clouds at 
one, zero thousand, broken clouds at two zero thousand, visibility unrestricted. 
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Winds Aloft Forecast: 

030 060 090 

ORF 1920 2025+5 2130+2 

RIC 2020 2125+4 2130+1 

Winds aloft forecast for the Norfolk, and Richmond, Virginia areas after one, seven, zero, 
zero universal coordinated time. 

Norfolk at three thousand: wind one, niner, zero at two, zero. 

At six thousand: wind two, zero, zero at two, five, temperature plus five. 

At niner thousand: wind two, one, zero at three, zero, temperature plus two. 

Richmond at three thousand: wind two, zero, zero at two, zero. 

At six thousand: wind two, one zero at two, five, temperature plus four. 

At niner thousand: wind two, one, zero at three, zero, temperature plus one. 

NOTAMS: 

No Current NOTAMS Listed. 

ATC Delays: 

NONE 

ATC request PIREPS for turbulence or other conditions along your route of flight. Con- 
tact Flight Watch or Flight Service. 

Washington Flight Watch is available with En-route Flight Advisory Service to update 
your weather briefing on 122.0MHz. Leesburg Flight Service station is available on 
122.2 MHz for weather briefings and other in-flight services. 
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Appendix G. Cockpit Research Facility Description 


The RTI/NASA Cockpit Research Facility (CRF) was configured for the study as a con- 
ventionally equipped aircraft with the addition of a display of FISB information. The 
CRF consists of three major subsystems (as illustrated in Figure G-l): 

• Rapid Prototype Simulator Cab - Consists of the cockpit mockup with controls, in- 
struments, radios and indicators. A Closed Circuit Television (CCTV) camera is 
mounted behind and above the pilots’ left shoulder to provide live images from the 
cockpit to the Scenario Controller and Observer Position. 

• Scenario Controller and Observer Position - Consists of the master control station, 
which is used for scenario generation, selection, monitoring and recording of flight 
progress. Provides the operator and experiment observer with displays of all control 
positions, radio and instrument switch positions, instrument displays and the Out-the- 
Window (OTW) (as presented to the subject pilot). A weather data display of 
NEXRAD images is provided for the scenario controller and for the observer to track 
the flight’s progress relative to the weather. A video image of the cockpit from the 
CCTV camera is provided for the observer to monitor the subject pilot’s actions. 

Live audio of all radio transmissions between the pilot and the NAS (controller, 

Flight Watch, ATIS, etc.) are available to the scenario controller and the observer. 

An intercom audio network is provided which allows private conversations between 
the scenario controller, observer and air traffic controller positions. Simulated radio 
transmissions between the pilot and air traffic controller are also enabled over the 
same intercom system. All intercom traffic is recorded on the audio track that ac- 
companies the video recording made from the CCTV camera. 

• ATC Controller Position - Consists of a custom ATC station developed for perform- 
ing experiments of this type and a weather display that shows the latest NEXRAD 
images to track the flight’s progress relative to the weather. Current pilot-selected 
COM frequencies are displayed so that the ATC controller can verify that the pilot is 
contacting ATC on the correct frequency before responding to an initial contact. 

A high-level diagram of the major system components is illustrated in Figure G-2. These 
major components are described in the following paragraphs. 
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Figure G-1. Cockpit Research Facility 



Airframe 
Simulation 
Model of 
Piper Malibu 



— bfl- 
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SGI Onyx 10000 
Infinite Reality, 
OS-IRIX 6.2, 3-Display 
Head C onfiguration 


Simulation jj 
Control and j: 
Flight Path jj 
Monitoring 


Ethernet 

Hub 




P60 PC, 
WIN 95 


SGI Indigo 2 
OS-IRIX 5.3 


Cab Instruments and Controls 


•Controls 

-Yolk 

- Rudder Pedals 

- Toe Brakes 

- Throttle 

- Mixture 

- Prop 

•Flight Instruments 

- Attitude 
-VSI 

- Airspeed 

- Turn & Bank 

- Directional Gyro 

- Altimeter 
-VOR 
-ILS 

•Nav & Com Radios 

•Altimeter 

•OBS 

•Engine Instruments 
-MAP 

- CHT 
-EGT 

- Oil Temp 
-RPM 
-Fuel 

- Fuel Flow 
•Switches 

- Pitot Heat 

- Landing Lights 

- Rotating Beacon 

- Fuel Select 

- Gear 

- Flaps 
-Mags 

- Fuel Pumps 


Figure G-2. Cockpit Research Facility System Block Diagram 


98 














Simulator Cab Description 

The simulator cab is a two-seat cockpit mockup designed for single-pilot IFR operations. 
The basic ergonomic structure of the mock-up is patterned after a generic GA airplane in 
terms of the relative placement and types of controls and instruments, instrument panel 
width and height, and seat placement. The pilot’s position is outfitted with complete 
controls including a yoke, rudder pedals, instruments, switches and indicators as de- 
scribed in the paragraphs below. 

The center console holds the radios and throttle quadrant. Both cockpit positions are out- 
fitted with headsets and an intercom system that allows the pilot to communicate with a 
passenger and with simulated Air Traffic Control. The primary out-the-window view is 
provided by a 37-inch monitor mounted directly in front of the pilot, approximately at the 
position of the aircraft nose. 

Controls, Instruments and Indicators Description 

The controls, instruments and indicator configurations available are typical of those 
found in an IFR-equipped aircraft as shown in Figure G-3. The characteristics for this 
configurations are summarized below. 

All instrument panel round dial indicators are rendered on flat panel liquid crystal dis- 
plays (LCDs). The 14-inch diagonal LCDs provide enough display area to fully render 
the standard instrument “T” configuration with other supplemental indicators as well. A 
second 14-inch diagonal LCD provides display area for navigation instruments and en- 
gine parameter indicators. Table G-l lists the types of instruments rendered in the cock- 
pit. All instruments provide the operational performance required by the Federal Avia- 
tion Administration Federal Aviation Regulations, Society of Automotive Engineers 
Aerospace Standards and RTCA, Inc. performance specifications as applicable to simu- 
lation. 
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Figure G-3. Instrument Panel, Controls and Indicators 


Table G-1. Instruments and Indicators in the Instrument Panel 


Flight Control 
Instruments 

Navigation / 
Communications 
Instruments 

Engine Monitoring 
Instruments 

Indicators 

• Attitude 

• Airspeed 

• Altitude 

• Turn and Bank 

• Slip & Skip 

• Gyro Stabilized 
Direction 

• Vertical Speed 

• Autopilot 

• VOR / DME 
Display #1 with 
ILS Localizer 
& Glideslope 

• VOR / DME 
Display #2 with 
ILS Localizer 
& Glideslope 

• NAV Radio #1 

• NAV Radio #2 

• COM Radio #1 

• ADF 

• Transponder 

• Manifold Pres- 
sure (MAP) 

• Engine RPM 

• Fuel Quantity 

• Fuel Flow 

• Oil Pressure 

• Exhaust Gas 
Temperature 
(EGT) 

• Cylinder Head 
Temperature 
(CHT) 

• Trim Position 

• Flap Position 

• Gear Position 

• Autopilot Ad- 
justment 
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The control yoke provides the pilot with an electric trim button, push-to-talk switch for 
the intercom system, chronometer for time calculations as well as a full range of control 
movement for controlling the flight path of the airplane. Activation of the electric trim 
button moves the yoke in or out to relieve the control forces. The current trim position is 
displayed on a trim indicator in the instrument panel. A display indicator on the PFD no- 
tifies the pilot that the autopilot is engaged and if it needs additional trimming. 

Visual System and Displays Description 

A Silicon Graphics Onyx 10000 is used to generate the instrument panel gages and the 
Out-the Window (OTW) scene for the pilot as depicted in Figure G-3. A rapid 
prototyping tool is used to develop and render the regulation-compliant gages in appear- 
ance and perfomiance. Round dial instruments are rendered on two 14-inch diagonal ac- 
tive matrix Liquid Crystal Displays (AMLCDs), each having an addressable resolution of 
1024 pixels by 768 lines. 

The OTW scene is a photo-textured presentation rendered at a 40 degree horizontal by 30 
degree vertical field-of-view and displayed on a 37-inch monitor at an addressable reso- 
lution of 1280 pixels by 1024 lines. The monitor is positioned so that active display area 
subtends approximately 40 degrees horizontal to the pilot’s eye point. Both instrument 
displays and OTW scene are rendered at a 30 Hz frame rate with a 70 Hz display refresh 
rate. 

A visual terrain database for the state of Virginia contains six major airports at which 
takeoffs and landings can be made. Another 24 airports are rendered at photographic 
quality to facilitate pilotage along several routes between NASA Langley, Newport 
News/Williamsburg Blacksburg, Richmond, Manassas, Washington National and NASA 
Wallops Island runways. The environmental conditions be varied to achieve any mete- 
orological conditions required, i.e. overcast, low RVR, cloud decks, etc. 

Data Acquisition System Description 

The data acquisition system is used to collect information about the pilot’s control inputs 
and switch actions, format the data, and transfer the data to the SGI Onyx for processing 
in the simulation models. 

A data acquisition controller system is hosted in a Pentium 60-based PC as depicted in 
Figure G-2. The data acquisition controller contains a microcontroller that performs all 
input / output (I/O) operations with the hardware in the simulator cab. Operations per- 
formed by the controller include: 

• Acquiring analog control position information 

• Performing the analog to digital (A/D) conversions on control position infor- 
mation 

• Acquiring switch position discreets 

• Driving indicators in the cab (e.g. Gear Position Indicators, Outer Marker , 
Middle Marker) 
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• Acquiring frequency selections set in the COM, NAV 1, NAV 2, DME, ADF, 
transponder and autopilot interfaces in the radio stack located in the cockpit 
center console 

• Acquiring the OBS 1 & 2, DG and Barro Altimeter knob settings 

• Updating the frequency displays in the COM, NAV 1, NAV 2, DME and ADF 
radios 

Simulation Control, Monitoring and Recording 

All aspects of the simulation are controlled by the simulation control and flight path 
monitoring process running in the Silicon Graphics Indigo as shown in Figure G-2. The 
simulation control process initializes the simulation models in the SGI Onyx, performs 
real time data display and data collection capture of various flight parameters for later 
analysis, and presents a plan view of the aircraft’s position during operation of the simu- 
lation, similar to an ATC console. The system operator uses the simulation control to 
select various scenarios, position / reposition aircraft model and monitor scenario prog- 
ress. 

Table G-2 lists the real-time parameters displayed at the operators station during system 
operation. Table G-3 lists the data dictionary of parameters available for collection and 
reduction. 

Table G-2. Real-time Parameters Displayed During Operations 


Parameter 

Parameter 

Parameter 

Parameter 

Airspeed (A/S) 

• Calculated A/S 

• Indicated A/S 

• True A/S 

Aerodynamic Coefficients 

• CL Total Lift 

* CD Total Drag 

* CY Total Side Force 

• CM Total Pitching Moment 

* CR T otal Rolling Moment 

• CN Total Yawing Moment 

Control Surface Deflection 

• Elevator 

• Rudder 

• Aileron 

• Aileron Trim 

• Rudder Trim 

• Trailing Edge Flaps 

• Left Spoiler 

• Right Spoiler 

Ground Contact Conditions 

(Landing) 

• Rate of Decent 

• Bank Angle 

• Side & Vertical Forces 
on Nose Gear 

• Side & Vertical Forces 
on Left Gear 

• Side & Vertical Forces 
on Right Gear 

Ground Speed 

Altitude 

• Pressure 

• AGL 

Position 

• Latitude 

• Longitude 

• Heading 


Aircraft Body Angles 

• Pitch 

• Roll 

• Yaw 

• Angle of Attack OC 

• Slide Slip |3 

Atomspheric 

• OAT 

• Air Pressure 

• Barro Pressure 

Weight & Balance 

• Gross Weight 

• Payload 

• Total Fuel 

• CG relative to 35% 
Mean Aerodynamic 
Chord (MAC) 



Engine Thrust 

Rate of Climb 
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Table G-3. Dictionary of Recordable Parameters and Inducible Faults 


Parameters 

Aerodynamic Model 

Altitude 

Altitude Pressure 

Aileron Position 

Column Force 

Elevator Position 

Elevator Trim position 

Rudder Position 

Stall Buffet 

Wheel Force 


Altimeter Setting 

Indicated Rate of Climb 

Roll Attitude 

| On Ground Status 

Weight on Gear 

Weight on Nose Wheel 



Angle of Attack 

Side Slip Angle 

Pitch Angle 

Pitch Acceleration 

Actual Rate of Climb 

Roll Angle 

Calculated Airspeed 

Ground Speed 

True Airspeed 


| Atmosphere | 

| Atmospheric Pressure 

OAT degrees C 

OAT degrees F 

Ambient Air Pressure | 

Autopilot - 22 Parameters 

Circuit Breakers - 20 Breakers 

ICE - Induced, Pitot Head, etc - 19 Parameters 

Gear - True Gear Positions, Nose, Left, Right 

1 nay I 

RMI / ADF Indicator 

DME Distance 

DME Speed 

DME Time 

DME Mode Switch 

OBS 1, OBS 2 

CDI 1.CDI2 

Glide Slope 1, Glide Slope 2 


Outer Marker 

Middle Marker 


S Induced Faults | 

Runaway trim 

Autopilot Pitch Axis Failure 


Autopilot Soft Roll 

Autopilot Circuit Breaker 

Dead Battery 


NAV 1 Failure 

Nav 2 Failure 


Vacuum System I 

| Position 

Standby Vacuum On 

Vacuum Hg 

| Latitude 

| Longitude 

Vacuum Enunciator 

Pump Switch 

S Controls | 

| Throttle Position 

Prop Position 

Mixture Position 

1 

| Weight and Balance i 

| Center of Gravity 

| Long Load Force 

| CG % MAC ! 







Air Traffic Management Console 

The Air Traffic Management (ATM) console is used during the conduct of research proj- 
ects to provide a more realistic environment for the subject pilot(s) involved in the re- 
search. During investigations, the station is manned by an experienced Air Traffic Con- 
troller. The ATM station receives data from the simulator and presents it on the ATM 
Station monitor in a manner sufficient to support the ATM functions required of the Air 
Traffic Controller. 

The screen consists of “radar image data” and associated mapping features, Figure G-4. 
System controls and informational data is presented on the side and top of the display. 
The operator may zoom in to a 1 mile scale (used for ground control) to a 100 mile scale 
(approach, departure, and enroute functions). Features that can be displayed during the 
operation of the ATM Station include: intersections (with and without names), airports 
(with and without names), runways (utilized during approaches), taxiways (utilized for 
ground control), VORs (with and without names), and special use airspace. The display 
is centered upon the selected airport (currently PHF, RIC, LFI, or WAL). Future imple- 
mentations include the display and manipulation of the flight paths of multiple aircraft. 
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Figure G-4. Air Traffic Management Console Display 


Aerodynamic Simulation Model 

The simulation model is a 6 Degree of Freedom (DOF) aerodynamic model that is table 
driven to provide the performance characteristics for the Piper Malibu PA46-310P. The 
Piper Malibu represents a high perfomiance single engine GA with a cruising speed of 
170 knots. The simulation model is executed in the SGI Onyx, based on the pilot inputs 
collected through the data acquisition system. It is computed in 3 parts: fast rate (30 
Hertz) coefficients, medium rate (15 Hertz) coefficients and slow rate coefficients (7.5 
Hertz). 

The aerodynamic coefficients in the simulation model incorporate the non-linear charac- 
teristics of an operational airplane. These adjustments give the simulation model more 
realistic longitudinal handling characteristics and make it possible, for example, to flare 
the airplane to a maximum lift stall at the touchdown point if desired. 
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Crosswind Model 

A cross wind model is available that can direct a cross wind over a large range from any 
direction. A 15 kt cross wind is shown relative to the nose of the aircraft in Figure G-5. 



Figure G-5. Cross Wind Model 


Turbulence Model 

A basic turbulence weather model is included in the simulation. As the aircraft ap- 
proaches a weather system the level of turbulence can be increased based on the overall 
level of convective weather and distance from the weather. 

Four levels of turbulence are calculated, from mild (level 1) to heavy (level 4). For each 
level, a random turbulence factor is added into the wind velocity for each of the respec- 
tive axis wind velocities. All instruments react to the turbulence in a manner reflecting 
the movement of the airframe through the airmass, i.e. rapid fluctuations in airspeed, ver- 
tical speed, attitude, heading, etc. 

The autopilot is programmed to disengage when the aircraft flies into areas of level three 
turbulence or higher. Attempts to reengage the auto pilot while in this level of turbulence 
will result in an automatic disengagement within 10 seconds. 
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Appendix H. Pre-Flight Briefings and Simulator Training 


Contrtol Group Preflight Briefing 
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Subject Pilot Schedule 

Time 

Activity 

0:20 

Introduction 

1:30 

Simulator Familiarization 

0:10 

Break 

0:30 

Flight Planning 

1:30 

Flight Experiment 

0:30 

Debriefing 


Today’s Flight Mission 



Situation 

• A diabetic patient is in urgent need of insulin at 
Wallops Island on eastern shore of Virginia. 

• The insulin is vital to survival of the patient. The 
longer the delay, the greater the likelihood that 
patient will not survive, or at best, suffer serious 
complications 

•Potentially fatal complications include Diabatic 
Ketoacidosis (DKA). One therapy for DKA 
includes treatment with sodium bicarbonate. 
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Today’s Mission 

(Continued) 


• RTI Medical Services, Inc. is to deliver insulin 
to the Wallops Island airport from NNWB 
airport, stopping enroute to pick up sodium 
bicarbonate at Richmond, Va airport. 

(The sodium bicarbonate medicine will be driven 
out to A/C at end of runway) 

• Departure of the RTI Medical Services flight is 
1900 hours this evening. 




Simulation Hardware Configuration 
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Simulation Cockpit Configuration 


Weather Information Sources 

ATIS 

Flight Service Station 
Flight Watch 

Virginia AWOS/ASOS Reports via radio 
Air Traffic Control (IAW normal NAS procedures) 
Tower 
Departure 
Enroute 
Approach 


Simulation Cockpit Configuration 



Electric Trim Switch 
Yoke 
Gear Switch 
Flap Switch 
Gear Indicators 
Throttle Quadrant 
Radio Stack 


A/S - Airspeed 
ATT - Attitude 
ALT - Altitude 
DG - Directional Gyro 
VSI - Vertical Speed Indicator 
ADF - Automatic Direction Finder 


Taxi Lights Switch 
Landing Lights Switch 
Rotating Beacon Switch 
NAV Lights Switch 
Fuel Pump Switch 
Pitot Heat Switch 
Trim Indicator 


Oi - Oil Temperature and Pressure 
MAP - Manifold Pressure 
TEMP - EGT&CHT 
Fuel - Fuel Quantity 
Flow - Fuel Flow 
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■ Simulation 

Cockpit 

Radio 

Stack 
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Treatment Group Preflight Briefing 



F A A/N AS A/RTI 
Flight Information Services 
Data Link (FISDL) 
Experiment 

June, 2000 
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Subject Pilot Schedule - 

Time 

Activity 

0:20 

Introduction 

1:30 

Simulator Familiarization 

0:10 

Break 

0:30 

Flight Planning 

1:30 

Flight Experiment 

0:30 

Debriefing 


Today’s Flight Mission 



Situation 

• A diabetic patient is in urgent need of insulin at 
Wallops Island on eastern shore of Virginia. 

• The insulin is vital to survival of the patient. The 
longer the delay, the greater the likelihood that 
patient will not survive, or at best, suffer serious 
complications 

•Potentially fatal complications include Diabatic 
Ketoacidosis (DKA). One therapy for DKA 
includes treatment with sodium bicarbonate. 
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Today’s Mission 

(Continued) 


• RTI Medical Services, Inc. is to deliver insulin 
to the Wallops Island airport from NNWB 
airport, stopping enroute to pick up sodium 
bicarbonate at Richmond, Va airport. 

(The sodium bicarbonate medicine will be driven 
out to A/C at end of runway) 

• Departure of the RTI Medical Services flight is 
1900 hours this evening. 




Simulation Hardware Configuration 




114 





Simulation Cockpit Configuration 



Weather Information Sources 

• ATIS 

• Flight Service Station 

• Flight Watch 

• Virginia AWOS/ASOS Reports via radio 

• Air Traffic Control (IAW normal NAS procedures) 

Tower 

Departure 

Enroute 

Approach 

• Data Link Flight Information Services (FISDL) Display 



Simulation Cockpit Configuration 
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■ Simulation 

Cockpit 

Radio 

Stack 
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Zoom 


Graphics 


Menu Off 


50 mile scale, graphics only mode 
just showing airports in gray boxes 
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Softkeys 


Menu Display 
Mode: 

Off 


GPS Mode 
Off 
On 
Free 


Brightness 


Modes 


Display Mode: 
Graphics 
METAR 
NEX/METAR 


Movement Mode: 
Zoom 
Scroll 
Crosshair 


Joystick 


25 mile scale, graphics only mode. 
Showing airports in gray boxes, and 
NavAids in blue circles (navaids only 
shown in scales of 25 miles or less) 
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10 mile scale, METAR mode. 
Showing airports in gray boxes, 
NavAids in blue circles, and graphical 
METARs 
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N EXRAD 05/25 1 2:57j| 


NEX/MTR 


NEXRAD Intensity Levels 


Lt. Green WM 
Dk. Green H 
Yellow □ 
Dk. Yellow !■ 
Red H 
Magenta WM 


25 mile scale, NEXRAD/METAR mode. 
Showing airports in gray boxes, 

NavAids in blue circles, graphical 
METARs and NEXRAD image with time stamp 
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Simulator Briefing and Training 

Both pilot groups were provided the opportunity to complete a practice flight in the 
simulator. The researcher/trainer guided the subject pilot through maneuvers to acquaint 
them with the operation and performance of the simulator. Additional instruction was 
given to the experimental group of pilots on the weather display. 

AWIN Experiment Simulator Familiarization Flight Syllabus 

1. General explanation of cockpit layout: 

Primary flight instruments 

Secondary instruments 

Sub panel controls and systems 

Yoke controls 

Radios 

Autopilot 

Intercom 

Charts 

2. Checklist explanation 

3. Engine start and taxi 

4. Runnup and system check 

5. Normal takeoff and climb 

6. Level off at 3000 feet (± 100 feet) 

7. Shallow and steep banked turns to a heading (± 10 degrees) 

8. Autopilot: 

Engage/ disengage 
Pitch modifier 
Altitude hold 
Altitude modifier 
Heading hold 

9. VOR operation (on AP) 

10. Use of weather display (on AP) 

11. Vectors to normal VFR landing, touch-and-go (±10 kts) 

12. Go-around (± 10 kts) 

13. Vectors to IFR approach and landing (second landing if required) 
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Appendix I. Observer Form 

AWIN EXPERIMENT FLIGHT RECORD 


Pilot Name: Subject # 

Experimental Group 

Display Condition: WITH / WITHOUT WX DISPLAY (Circle one) 
Risk Score Wx Score Fit Hrs Actual IMC 

CHECKLIST A: RICHMOND DECISION 


BEHAVIORAL RECORD 

1. DIVERT TO WALLOPS 

2. HOLD 

2.1 >10 MINS & SENT TO WAL APPR 

(Time in hold = mins) 

2.2 <10 MINS & DIVERT TO WAL 

2.3 <10 MINS & BEGINS APPROACH 

3. MAKE APPROACH 

3.1 GO AROUND 

3.2 ATTEMPT LAND -BUT SENT 

ON TO WAL 

4. ABORT MISSION 

4.1 RETURN TO PHF 

4.2 OTHER 


INFORMATION SEARCH RECORD 

1. R/T INQUIRY 

1.1 PHF DEP 

1.2 RICHMND 

1.3 FSS 

1.4 FLT WATCH 

2. AUTOM SERVICES 

2.1 ATIS 

2.2 AWOS/ASOS 

3. WX DISPLAY 

3.1 TEXT MET AR 

3.2 GRAPHIC 

3.3 CHANGE ZOOM 

3.4 GPS LOC/NLOC 

4. OTHER INFO SOURCE 
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WALLOPS DECISION 


BEHAVIORAL RECORD 

5. PENETRATED WX DIRECT 

WALLOPS 

5.1 CHANGED ALTITUDE 

5.2 REMAINED AT FLTPLN ALT 

6. DIVERTED AROUND WX 

6.1 TO THE SOUTH 

6.2 TO THE NORTH 

7. ENTERED HOLD PENDING WX 
CHANGE / UPDATE 

8. ABORTED 

10.1 TURNED BACK FOR 
RICHMOND 

10.2 TURNED BACK FOR PHF 


INFORMATION SEARCH 
RECORD 

1. R/T INQUIRY 

1.1 RICH DEP 

1.2 WASH 

1.3 WALLOPS 

1.4 FSS 

1.5 FLT WATCH 

2. AUTOM SERVICES 

2.2 ATIS 

2.3 AWOS/ASOS 

3. WX DISPLAY 

3.1 TEXT METAR 

3.2 GRAPHIC 

3.3 CHANGE ZOOM 

3.4 GPS LOC/NLOC 

4. OTHER INFO SOURCE 
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Appendix J. Immediate Reactions Questionnaire 


Please use the following scales to rate how much you agree or disagree with the state- 
ments offered. If some items appear to overlap, do not be concerned, but attempt to an- 
swer each on its own terms. There are no “right” or “wrong” answers, nor any agenda of 
preferred responses being sought by the researchers. 

1. a. An “ownship” aircraft symbol, cross-hairs or some similar position indication 
would be a useful addition to the weather display. (Circle One) 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

b. Without some way to ascertain position relative to other displayed features the dis- 
play’s usefulness is significantly compromised. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

2. a. I took the medical emergency scenario seriously, in the sense that I factored the 
emergency into my decision making. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

b. While taking the medical aspect of the flight seriously, it did not figure at the fore- 
front of my in- flight decision making. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

3. My knowledge that this was all a simulation, that nobody’s health or welfare was 
really at stake, influenced the way in which I managed the flight. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


4. An advantage of the onboard Weather Display was showing the weather in real-time, 
that is, as it actually was at that moment? 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


125 



5. I attribute much of my decision making to my interpretation of the Weather Display. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

6. I tried to systematically sample all sources of weather information open to me. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

7. I used the Weather Display but felt the need to cross-check or verify my conclusions 
from conventional weather data sources (ATC, etc.) 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

8. a. I felt comfortable with the autopilot, in terms of understanding its use & opera- 
tion. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

b. Without autopilot my completion of the flight would have been compromised. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 


9. a. The degree of validity of the weather data appearing on-screen was a factor I felt I 
held in mind as I flew. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 

b. I have been monitoring the weather display time stamp very regularly in my in- 
strument scan. 

Disagree Disagree No Opinion Agree Agree 

Somewhat Somewhat 
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10. At the time of my arrival at Richmond ‘s Airport I knew that there was a storm 
(check one) 

a. About 10 nm North West of the field 

b. About 5 nm North West of the field 

c. Near the fied 

d. Right at the field 

11. At the time I was en route to Wallops I saw, across my path of direct flight, what I 
took to be. 

a. a penetrable storm 

b. a navigable opening between convective cells 

c. a non navigable opening between cells 

d. a wall of convective activity requiring diversion. 
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Appendix K. Structured Interview Guide 


The interviewer follows the procedure set out on this form, opening with the following 
explanatory statement to the pilot: 

"In this debriefing we first give you a short questionnaire to complete while your 
memory is fresh. Then we conduct what we call a structured interview, which 
means we ask everyone the same set of questions in pre-determined categories 
(what, why, & so on), using this checklist form. The form’s in 3 parts. We’ll begin 
with Part A, which is just verifying the purely factual aspects of your flight, without 
getting into reasons, or other commentary. That comes under B. Your own, open- 
ended feedback on the flight is very important to us & we’ll cover that last in Part 
C." 


A. BEHAVIORAL RECORD CONFIRMATION 

"We show that you ...." 

The interviewer here uses the previous printed checklists and verifies i. the be- 
havioral record, & ii. the information search record, then completes this form. 

□ BEHAVIORAL RECORD CONFIRMED 

□ INFORMATION SEARCH RECORD CONFIRMED 

□ BEHAVIORAL RECORD DISCREPANCIES (Enter nature below) 


□ INFORMATION SEARCH RECORD DISCREPANCIES 
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“O.K., that’s the verification of what took place. Now we move on to why questions.” 


EL DECISION RATIONALE 


“Going through the flight again, looking at it phase by phase in sequence, I’d like 
you to tell me why you chose to adopt certain courses of action and not to select 
other options. This might entail your repeating certain rationales (as some options 
overlap others), and sometimes it may feel like you are having to state or restate the 
obvious. This doesn’t matter - it just happens sometimes in structured interviews. 
So please bear with us. Doing it this way helps the researchers cross-check the data, 
identify subtleties, and avoid making assumptions or inferences about what was or 
was not obvious ” 


RICHMOND DECISION 


1. DIVERT TO WALLOPS 

“First, why did you / didn’t you divert to Wallops on the leg to Richmond?” 


2. HOLD 

“O.K., why did you / didn’t you stay put in a holding pattern?” 


3. MAKE APPROACH 

“Why did you choose to / choose not to make the approach into Richmond?” 


“Why did you / didn’t you go around ?” 
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“Why did you / didn’t you attempt to land?” 


4. ABORT MISSION 

“Why did you / didn’t you abort the mission?” 


“Why did you / didn’t you return to PHF?” 


OTHER (Optional) 

“Why did you choose to [ insert any other behavior not anticipated in the form ]?” 
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EN ROUTE WALLOPS DECISION 


1 . PENETRATION OF WX EN ROUTE TO WALLOPS 

“Why did you choose to / not choose to penetrate the weather en route to Wallops?” 


2. DIVERSION AROUND WALLOPS 

“ Why did you/didn’t you divert north around the weather en route to Wallops? 
(e.g. pending a wx update or change of wx.) 


“Why did you/didn’t you divert south around the weather en route to Wallops?” 


3. ENTERED HOLD 

“Why did you / didn’t you enter a holding pattern en route to Wallops?” 


4. ABORT ATTEMPT TO REACH WALLOPS 

“Why did/didn’t you abort efforts toward the Wallops destination & return to 
Richmond / PHF?” 
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OPEN COMMENTS & QUESTIONS (OC & Q) SECTION 


“Thanks. That’s the structured or formal part over with. Now we can go on to the 
Open Comments & Questions stage where you can give us your feedback & we all 
pitch in.” 

Note to ourselves: The OC&Q PERIOD HAS TWO MAIN FUNCTIONS 

1. To enable us to go back & correct, expand or qualify earlier responses in the ques- 
tionnaire or debrief. The subject’s responses may not all have seemed consistent & 
these apparent inconsistencies can be explored & reconciled. The data we collect 
here is non-quantitative but has an important role - it becomes prime content for the 
discussion sections of the TR . So notes should be made at appropriate points in the 
questionnaire or debriefing guide. Major new points should be appended below. 

2. To enable the subject pilot and the research team to identify new issues, contribute 
new views, perspectives, and “takes” on the issues that may not have been anticipated 
in the formal debrief, or that may involve dissenting or alternative interpretations of 
events, etc. Important points not captured elsewhere should be appended below. 
Again, the point is to secure context & richness for the TR. 
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Appendix L. Weather Display Questionnaire 


The following is the questionnaire given to the pilots that flew with the weather display. 
It includes basic flight experience questions, previous in-flight weather display use, and 
their open opinions about the weather display. 

Subject Pilot #: Date: 

1 . Approximately how many total hours do you have? 

2. Approximately how many actual instrument hours do you have? 

3. Approximately how many simulator hours do you have? 

4. Approximately how many instrument hours do you have in the last 90 days ? 


5. What ratings do you have? (circle as many as apply) 

Private Commercial ATP Glider Airship Sea 
Instrument CFI CFII MEI Helicopter A&P IA 

6. What type of aircraft do you have most of your experience in? 


7. Have you ever used a datalinked in-flight weather display system in a flight? 
(not including onboard radar or Stormscope) 

If yes, how many flights do you have with it? 
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8. If you answered yes to the last question, how many times have you used that data- 
linked in-flight weather display system to make actual weather judgements? (In- 
stead of just experimenting with the display). 


9. Have you had any training in weather interpretation other than basic pilot training (for 
example, courses in meteorology)? If so, what? 


10. What is your usual method of obtaining a pre-flight weather briefing? 
(DUATS, FSS phone, etc.) 


11. Have you tried other alternate methods of weather briefings, and what was your expe- 
rience? 


12. Do you feel that you took the simulator as seriously as a real airplane? 


13. In using this weather display today, did you find the operation straightforward? If not, 
what operations of this weather display did you find difficult? 
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14. In using this weather display today, did you find the graphical METAR symbology 
useful? If not, what features did you find difficult? 


15. In using this weather display today, did you find the textual METAR presentation 
useful? If not, what features did you find difficult? 


16. Considering your use of the weather display today, would you like to see any addi- 
tional features or change any existing features? 


17. What features of the weather display did you find helpful in updating your route? 


18. How did you determine the age of the weather information? 


19. Were there any features about the weather display that caused you to cast doubt as to 
its usefulness in normal, real world, operation? 
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20. Did you find that the weather display increased or decreased your workload? 


21. How did the use of the autopilot help or hinder the use of the weather display? 


22. Did you find that the weather display increased or decreased your situational aware- 
ness? 


23. Have you ever been in a situation that you would not have placed yourself in, if you 
had an in-flight weather display? 


24. Can you think of any specific instances where you wished you had this type of cock- 
pit weather information? Please be brief. 


Thank you very much for participating in our study, we appreciate your help. 


138 



Appendix M. AWIN Study Questionnaire 


The following is the questionnaire given to the pilots that did not fly with the weather 
display. It includes basic flight experience questions and previous in-flight weather dis- 
play use. 

Subject Pilot #: Date: 

1 . Approximately how many total hours do you have? 

2. Approximately how many actual instrument hours do you have? 

3. Approximately how many simulator hours do you have? 

4. Approximately how many instrument hours do you have in the last 90 days ? 


5. What ratings do you have? (circle as many as apply) 

Private Commercial ATP Glider Airship Sea 
Instrument CFI CFII MEI Helicopter A&P IA 

6. What type of aircraft do you have most of your experience in? 


7. Have you ever used a datalinked in-flight weather display system in a flight? 
(not including onboard radar or Stormscope) 

If yes, how many flights do you have with it? 
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8. If you answered yes to the last question, how many times have you used that da- 
talinked in-flight weather display system to make actual weather judgements? (Instead of 
just experimenting with the display). 


9. Have you had any training in weather interpretation other than basic pilot training 
(for example, courses in meteorology)? If so, what? 


10. What is your usual method of obtaining a pre-flight weather briefing? 
(DUATS, FSS phone, etc.) 


11. Have you tried other alternate methods of weather briefings, and what was your expe- 
rience? 


12. Do you feel that you took the simulator as seriously as a real airplane? 


Thank you very much for participating in our study, we appreciate your help. 
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Appendix N. Air Traffic Control Scripts 

Communication Exchanges Between Pilot and ATC 


The following is a typical communication exchange for the mission. Each pilot deviated 
from this typical exchange, some more than others, but only to the extent of clarifying 
radio calls, routing changes and exchanges to gather weather information. 


AWIN EXPERIMENT: FIRST LEG 


SUMMARY: INSTRUMENT FLIGHT FROM NEWPORT NEWS/ 
WILLIAMSBURG (PHF) AIRPORT TO RICHMOND INTERNATIONAL (RIC) 
AIRPORT VIA DIRECT HOPEWELL V260 RICHMOND. SEVERE 
THUNDERSTORM APPROACHING RIC. PILOT TO DECIDE WHETHER TO 
CONTINUE APPROACH AND ATTEMPT LANDING AT RIC, HOLD 
AWAITING WEATHER IMPROVEMENT, OR BY-PASS RIC AND REQUEST 
CLEARANCE TO WALLOPS, VA (WAL) FLIGHT FACILITY. 


N73Y: (Tunes 128.65 for ATIS) 

ATIS: THIS IS NEWPORT NEWS WILLIAMSBURG INTERNATIONAL 
TOWER INFORMATION BRAVO. 1800 ZULU MEASURED CEILING 1000 
OVERCAST VISIBILITY 3 MILES. TEMPERATURE 14 DEWPOINT 12 WIND 
090 AT 10 ALTIMETER 29.92. LANDING AND DEPARTING RUNWAY 7. ILS 
RUNWAY 7 APPROACH IN USE. ADVISE YOU HAVE BRAVO. 


N73Y: Newport News clearance delivery, Malibu 2573Y ready for clearance. (121.65) 

ATC: MALIBU 2573 Y CLEARED TO RICHMOND VOR VIA DIRECT 
HOPEWELL V260 RICHMOND MAINTAIN 5000. SQUAWK 1424. 


N73Y: Roger, cleared to Richmond via direct Hopewell V260 Richmond maintain 5000. 
(Tunes 121.9) 

N73Y: Newport News ground control, N73Y ready to taxi, have information Bravo. 

ATC: N73Y, GROUND CONTROL, TAXI STRAIGHT AHEAD THEN LEFT TO 
RUNWAY 7. WHEN READY FOR TAKEOFF, CONTACT TOWER ON 118.7. 


N73Y: Malibu 73 Y, Roger. 
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(Tunes 118.7) 

N73Y: Tower, N73Y ready for takeoff. 


ATC: N73Y MAINTAIN RUNWAY HEADING FOR RADAR VECTORS 
HOPEWELL MAINTAIN 2000, EXPECT CLEARANCE TO 5000 WITHIN 5 
MINUTES AFTER DEPARTURE. CLEARED FOR TAKEOFF RUNWAY 7. 


N73Y: Malibu 73Y Roger, cleared for takeoff. 


(Departs) 

ATC: MALIBU 73Y CONTACT NORFOLK DEPARTURE CONTROL ON 124.9. 


(Tunes 124.9) 

N73Y: Norfolk departure control, this is N73Y climbing to 2000 on runway heading. 


ATC: N73Y ROGER, IN RADAR CONTACT. TURN LEFT PROCEED DIRECT 
HOPEWELL, CLIMB AND MAINTAIN 5000. 

N73Y: Malibu 73 Y Roger, Proceeding direct Hopewell. 

N73Y: Norfolk departure control, request permission to leave frequency for Richmond 
ATIS. 

ATC: N73Y FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON 
MY FREQUENCY. 

(N73Y tunes 1 19.15 for RIC ATIS) 

ATIS: THIS IS RICHMOND TOWER INFORMATION DELTA. 1910 ZULU 
MEASURED CEILING 200 OVERCAST VISIBILITY THREE QUARTERS 
THUNDERSTORMS MODERATE RAIN SHOWERS TEMPERATURE 14 
DEWPOINT 12 WIND 300 AT 10 ALTIMETER 29.92. ILS RUNWAY 34 
APPROACH IN USE. LANDING AND DEPARTING ON RUNWAY 34. ADVISE 
YOU HAVE DELTA. 


(Tunes 124.9) 


N73Y : Departure control, N73Y back on your frequency. 

ATC: MALIBU 73Y ROGER. 

ATC: MALIBU 73 Y CONTACT RICHMOND APPROACH CONTROL ON 134.7. 
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(Tunes 134.7) 

N73Y: Richmond approach control, this is Malibu 73Y. Have information Delta. 


ATC: N73Y, RICHMOND APPROACH CONTROL, ROGER, DESCEND AND 
MAINTAIN 2000. EXPECT VECTORS TO ILS RUNWAY 34 APPROACH. 


N73Y: N73Y, Roger, descending to 2000. 

ATC: N73Y DEPART HOPEWELL VOR HEADING 300 FOR A VECTOR TO 
ILS RUNWAY 34 FINAL APPROACH COURSE. 


N73Y: N73Y, Roger, depart Hopewell heading 300 for vector to ILS runway 34 ap- 
proach course. 

ATC: MALIBU 73 Y, 4 MILES SOUTHEAST OF KAFKA, MAINTAIN 2000 
UNTIL ESTABLISHED ON THE LOCALIZER, CLEARED FOR ILS RUNWAY 
34 APPROACH. CONTACT TOWER ON 121.1 PASSING KAFKA. 

N73Y: Malibu 73 Y, Roger, cleared for approach, tower 121.1 at Kafka. 


(Tunes 121.1) 

ATC BROADCAST: ATTENTION ALL AIRCRAFT IN RICHMOND AREA. 
LOW LEVEL WINDSHEAR ADVISORIES IN EFFECT FOR RICHMOND 
INTERNATIONAL AIRPORT. 

(ATC TO IMPROVISE HOLDING, RELEARANCE TO WALLOPS, OR MISSED 
APPROACH DEPENDING ON PILOTS DECISION/REQUEST WITH 
WEATHER ENCOUNTERED.) 
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AWIN EXPERIMENT: 2ND LEG 


SUMMARY: INSTRUMENT FLIGHT FROM RICHMOND (RIC) TO 
WALLOPS FLIGHT FACILITY (WAL) VIA RICHMOND DIRECT HARCUM 
DIRECT JAMIE VI MAGGO. AT PILOT’S REQUEST AFTER HOLDING OR 
EXECUTING A MISSED APPROACH AT RICHMOND. 

ATC: MALIBU 73 Y CLEARED TO THE MAGGO INTERSECTION VIA 
DIRECT HARCUM DIRECT JAMIE VI MAGGO. CLIMB AND MAINTAIN 
5000 CONTACT RICHMOND DEPARTURE CONTROL ON 126.4. 


N73Y: Malibu 73Y, Roger, proceeding direct Harcum climbing to 5000, changing to 
126.4. 

(Tunes 126.4) 

N73 Y: Richmond departure control Malibu 73 Y proceeding direct Harcum climbing to 
5000. 

ATC: N73Y, ROGER, IN RADAR CONTACT. 

N73Y: Departure control, N73Y, request permission to leave frequency for Wallops 
ASOS information. 

ATC: N73Y FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON 
MY FREQUENCY. 


(Tunes 119.175) 

WALLOPS AUTOMATED SURFACE OBSERVATION WIND 170 AT 6 
VISIBILITY 3 MILES. MEASURED CEILING 1000 BROKEN TEMPERA TURE 
23 DEWPOINT 16 ALTIMETER 29.92. 


(Tunes 126.4) 

N73Y: Richmond departure control, Malibu 73 Y back on your frequency. 

ATC: MALIBU 73Y, ROGER. 

ATC: (When N73Y is approximately 10-15 nm from weather cells depicted.) N73Y, I 
SHOW WEATHER AHEAD. ADVISE INTENTIONS. 


(Possible requests from N73Y as weather is encountered.) 
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N73Y: 1) Request deviation to south/north to avoid weather. 

2) What do you show for weather on my route of flight? 

3) Request vector around weather. 

4) Request a new route/ altitude to avoid weather. 

5) Request frequency change for Flight Watch or FSS. 


A TC: RESPOND TO SPECIFIC REQUEST, I.E: 

1) UNABLE TO APPRO VE DE VIA TION TO THE NORTH. RESTRICTED 
AREA 6609 IN USE. 

2) DEVIA TION TO THE SOUTH APPROVED. 

3) I SHO W HEA VY WE A THER ON YOUR PROJECTED FLIGHT PA TH. 

4) ROGER, TURN RIGHT HEADING FOR A VECTOR SOUTH OF 

WEATHER. 

5) FREQUENCY CHANGE APPROVED. ADVISE WHEN BACK ON MY 
FREQUENCY. 


ATC: N73Y CLEAR OF WEATHER FLY HEADING FOR VECTOR TO VI. 


N73 Y: N73 Y, Roger, turning to heading 

ATC: N73Y CONTACT PATUXENT APPROACH CONTROL ON 127.95. 


N73 Y: N73 Y Roger changing to 127. 95 
(Tunes to 127.95) 

N73Y: Patuxent approach control, this is Malibu 73 Y. 


ATC: N73Y THIS IS PATUXENT APPROACH CONTROL, EXPECT VOR/DME 
RUNWAY 10 APPROACH TO WALLOPS. ALTIMETER 29.92. 


N73Y: N73Y, Roger. 

A TC: N73 Y, DESCEND AND MAINTAIN 2000. 

N73Y: N73Y, Roger, leaving 5000 for 2000. 


A TC: (5 Miles south ofMaggo) TURN RIGHT HEADING 060 INTERCEPT THE 
SALISBURY 24.1 MILE ARC CLEARED FOR VOR/DME RUNWAY 10 
APPROACH. 
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N73Y: N73Y, Roger, heading 060 to the arc, cleared for VOR/DME Runway 10 ap- 
proach. 

ATC: MALIBU 7 3Y CONTACT WALLOPS TOWER ON 126.5. 

N73Y: Malibu 73Y , Roger changing to tower. 

(Tunes 126.5) 

N73Y: Wallops Tower, this is Malibu 73 Y on approach to runway 10. 

ATC: MALIBU 73Y, WALLOPS TOWER, WIND 170 AT 6, CLEARED TO LAND 
RUNWAY 10. 
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Appendix O. Enroute Weather Information Report Scripts 


The following weather report script was available to the Air Traffic Controller to be used 
as updated weather information. The reports were available while the mission was in pro- 
gress. But the information was only given to the pilot if requested. These reports were 
available through the Flight Service Station radio, Enroute Flight Advisory Service 
(Flight Watch) and Air Traffic Control frequencies. 

En-route Abbreviated Weather Reports 

AIRMET (WA) TANGO FOR OCNL MOD TURB BLO 060 for MD, VA and NC is 
current. 

ZDC CWA01 1855Z Valid Until 2100Z 
FROM CSN TO RIC TO DAN TO LYH TO CSN 

BKN AREA OF TSRA INCRG IN INTENSITY AND COVERAGE MOV EAST 

Washington Center Weather Advisory zero, one valid until two, one, zero, zero universal 
coordinated time.. From Casanova, Virginia to Richmond, Virginia, to Danville, Virginia, 
to Lynchburg, Virginia, to Casanova, Virginia. Broken area of thunderstorms and rain 
increasing in intensity and coverage, moving East. 

ZDC CWA02 1855Z VALID UNTIL 2100Z 
FROM SBY225025 TO RIC090050 

BKN LINE OF TSRA INCRG IN INTENSITY AND COVERAGE MOV LITTLE 

Washington Center Weather Advisory zero, two valid until two, one, zero, zero universal 
coordinated time. From two, five miles Southwest of Salisbury, Maryland to Five, zero 
miles East of Richmond, Virginia. Broken line of thunderstorms and rain increasing in 
intensity and coverage, moving little. 

RIC SP 1910Z M002 OVC 3/4TRW 58/55/9012G16/992/TSTM OVHD 
OCNL LGTCCCG 

Richmond International Airport special weather report one, niner, one, zero universal co- 
ordinated time. Measured ceiling, two hundered, overcast, visability 14, thunderstorm, 
moderate rain showers, temperature five, eight, dew point five, five, wind zero, nine, 
zero, at one two, gusting one six, altimeter two niner nine two, thunderstorm overhead, 
occasional lightning cloud to cloud, and cloud to ground. 
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LKU SP 1905Z MOOS OVC 1/2TRW+FG 57/57/2615G25/950/TSTM OVHD MOV E 
OCNL LGTCCCG 

Louisa County, Freeman Airport special weather report one, niner, zero, five universal 
coordinated time. Measured ceiling five hundred overcast, visibility one half, thunder- 
storm, heavy rain showers, fog, temperature five, seven, dew point five, seven, wind two, 
six, zero at one, five gusting two, five, altimeter two, niner, five, zero, thunderstorm 
overhead moving East, occasional lightning cloud to cloud, and cloud to ground. 

WAL SA 1846Z (Current) 

MVP SA 1846Z (Current) 

SBY SA 1845Z (Current) 

UUA: /OV RIC /TM 1900Z /FL 010-SFC /TP C210 /TB SVR /RM LLWS FA 

Urgent pilot report over Richmond, Virginia at one, niner, zero, zero universal coordi- 
nated time. From one thousand feet to the surface, a Cessna two, one, zero reported severe 
turbulence and low-level wind shear on final approach. 

UA: /OV SBY /TM 1905Z /FL 060 /TP BE55 /TB NEG /RM MANY BLD-UPS OVR 
BAY SW 

Pilot report over Salisbury, Maryland at one, niner, zero, five universal coordinated time. 
At six thousand feet, a Beech five, five reported negative turbulence, and many build-ups 
over the bay Southwest. 

UA: /OV RIC090050 /TM 1900Z /FL080 /TP PA46 /TB NEG /RM BLD-UPS OVR 
BAYN 

Pilot report five, zero miles East of Richmond, Virginia at one, niner, zero, zero universal 
coordinated time. At eight thousand feet, a Piper four, six reported negative turbulence 
and build-ups over the bay North. 

Satellite Imagery indicates solid Build-Ups forming throughout Central Virginia. 

Weather Radar indicates solid light to moderate precipitation with increasing areas of 
Heavy precipitation developing throughout Central Virginia moving eastward into the 
Richmond (RIC), and Mecklenburg-Brunswick (AVC)areas. 
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Appendix Q. NEXRAD Mosaic images 
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Appendix R. Aeronautical Information Manual FISDL Guidance 


Issue: 2/24/00 

7-1-10. FLIGHT INFORMATION SERVICES DATA LINK (FISDL) 

a. FISDL. Aeronautical weather and operational information may be displayed in the 
cockpit through the use of FISDL. FISDL systems are comprised of two basic types: 
broadcast systems and two-way systems. Broadcast system components include a terres- 
trial or pace-based transmitter, an aircraft receiver, and a cockpit display device. Two- 
way systems utilize transmitter/receivers at both the terrestrial or space-based site and the 
aircraft. 


1. Broadcast FISDL allows the pilot to passively collect weather and operational 
data and to call up that data for review at the appropriate time. In addition to text weather 
products, such as METAR's and TAPs, graphical weather products, such as radar com- 
posite/mosaic images may be provided to the cockpit. Two-way FISDL services permit 
the pilot to make specific weather and operational information requests for cockpit dis- 
play. 


2. FISDL services are available from three types of service providers. 

(a) Through vendors operating under a service agreement with the FAA 
using broadcast data link on VHF aeronautical spectrum (products and 
services are defined under subparagraph c). 

(b) Through vendors operating under customer contract on aeronautical 
spectrum. 

(c) Through vendors operating under customer contract on other than 
aeronautical spectrum. 


3. FISDL is a method of disseminating aeronautical weather and operational data 
which augments pilot voice communication with Flight Service Stations (FSS's), other 
Air Traffic Control (ATC) facilities or Airline Operations Control Centers (AOCC's). 
FISDL does not replace pilot and controller/flight service specialist/aircraft dispatcher 
voice communication for critical weather or operational information interpretation. 
FISDL, however, can provide the background information that can abbreviate and greatly 
improve the usefulness of such communications. As such, FISDL serves to enhance pilot 
situational awareness and improve safety. 

b. Operational Use of FISDL. Regardless of the type of FISDL system being used, ei- 
ther under FAA service agreement or by an independent provider, several factors must be 
considered when using FISDL. 
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1. Before using FISDL in flight operations, pilots and other flight crew members 
should become completely familiar with the operation of the FISDL system to be used, 
airborne equipment to be used, including system architecture, airborne system compo- 
nents, service volume and other limitations of the particular system, modes of operation 
and the indications of various system failures. Users should also be familiar with the 
content and format of the services available from the FISDL provider(s). Sources of in- 
formation which may provide this guidance include manufacturer's manuals, training 
programs and reference guides. 

2. FISDL does not serve as the sole source of aeronautical weather and opera- 
tional information. ATC, FSS, and, if applicable, AOCC VHF/HF voice is the basic 
method of communicating aeronautical weather, special use airspace, NOTAM and other 
operational information to aircraft in flight. FISDL augments ATC/FSS/AOCC services, 
and, in some applications, offers the advantage of graphical data. By using FISDL for 
orientation, the usefulness of any information received from conventional voice sources 
may be greatly enhanced. FISDL may alert the pilot to specific areas of concern which 
will more accurately focus requests made to FSS or AOCC for inflight briefings or que- 
ries made to ATC. 

3. The aeronautical environment is constantly changing; often these changes oc- 
cur quickly, and without warning. It is important that critical decisions be based on the 
most timely and appropriate data available. Consequently, when differences exist be- 
tween FISDL and information obtained by voice communication with ATC, FSS, and/or 
AOCC (if applicable), pilots are cautioned to use the most recent data from the most 
authoritative source. 

4. FISDL products, such as ground-based radar precipitation maps, are not appro- 
priate for use in tactical severe weather avoidance, such as negotiating a path through a 
weather hazard area (an area where a pilot cannot reliably divert around hazardous 
weather, such as a broken line of thunderstorms). FISDL supports strategic weather deci- 
sion making such as route selection to avoid a weather hazard area in its entirety. The 
misuse of information beyond it's applicability may place the pilot and his/her aircraft in 
great jeopardy. In addition, FISDL should never be used in lieu of an individual pre-flight 
weather and flight planning briefing. 

5. FISDL supports better pilot decision making by increasing situational aware- 
ness. The best decision making is based on using information from a variety of sources. 

In addition to FISDL, pilots should take advantage of other weather/NAS status sources, 
including, but not limited to, Flight Service Stations, Flight Watch, other air traffic con- 
trol facilities, airline operation control centers, pilot reports, and their own personal ob- 
servations. 

c. FAA FISDL. The FAA's FISDL system provides flight crews of properly equipped 
aircraft with a cockpit display of certain aeronautical weather and flight operational in- 
formation. This information is displayed using both text and graphic format. This system 
is scheduled for initial operational capability (IOC) in the first quarter of calendar year 
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2000. The system is operated by vendors under a service agreement with the FAA, using 
broadcast data link on aeronautical spectrum on four 25 KHz spaced frequencies from 
136.425 through 136.500 MHz. FISDL is designed to provide coverage throughout the 
continental U.S. from 5,000 feet AGL to 17,500 feet MSL, except in those areas where 
this is unfeasible due to mountainous terrain. Aircraft operating near transmitter sites will 
receive useable FISDL signals at altitudes lower than 5000 feet AGL, including on the 
surface in some locations, depending on transmitter/aircraft line of sight geometry. Air- 
craft operating above 17,500 MSL may also receive useable FISDL signals under certain 
circumstances. 

1. FAA FISDL provides, free of charge, the following basic products: 

(a) Aviation Routine Weather Reports (METAR's). 

(b) Special Aviation Reports (SPECI's). 

(c) Terminal Area Forecasts (TAF's), and their amendments. 

(d) Significant Meteorological Information (SIGMET's). 

(e) Convective SIGMET's. 

(f) Airman's Meteorological Information (AIRMET's). 

(g) Pilot Reports (both urgent and routine) (PIREP's); and, 

(h) Severe Weather Forecast Alerts (AWW's) issued by the FAA or NWS. 


2 . The format and coding of these products are described in Advisory Circular 
AC-00-45, Aviation Weather Services, and paragraph 7-1-28. Key to Aviation Routine 
Weather Report (METAR) and Aerodrome Forecasts (TAF). 

3 . Additional products, called Value-Added Products, are available from the ven- 
dors on a paid subscription basis. Details concerning the content, format, symbology and 
cost of these products may be obtained from the following vendors: 

(a) BENDIX/KING WxSIGHT 
Allied Signal, Inc. 

One Technology Center 
23500 West 105th Street 
Olathe, KS 66061 
(913)712-2613 
www.bendixking.com 
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(b) ARNAV Systems, Inc. 

16923 Meridian East 
P. O. Box 73730 
Puyallup, WA 98373 
(253) 848-6060 
www.amav.com 

d. Non-FAA FISDL Systems. In addition to FAA FISDL, several commercial vendors 
provide customers with FISDL on both the aeronautical spectrum and other frequencies 
using a variety of data link protocols. In some cases, the vendors provide only the com- 
munications system which carries customer messages, such as the Aircraft Communica- 
tions Addressing and Reporting System (ACARS) used by many air carrier and other op- 
erators. 


1. Operators using non-FAA FISDL for in-flight weather and operational infor- 
mation should ensure that the products used conform to the FAA/NWS standards. Spe- 
cifically, aviation weather information should meet the following criteria: 

(a) The products should be either FAA/NWS accepted aviation weather 
reports or products, or based on FAA/NWS accepted aviation weather reports or 
products. If products are used which do not meet this criteria, they should be so 
identified. The operator must determine the applicability of such products to flight 
operations. 

(b) In the case of a weather product which is the result of the application 
of a process which alters the form, function or content of the base FAA/NWS ac- 
cepted weather product(s), that process, and any limitations to the application of 
the resultant product should be described in the vendor's user guidance material. 

2. An example would be a NEXRAD radar composite/mosaic map, which has 
been modified by changing the scaling resolution. The methodology of assigning reflec- 
tivity values to the resultant image components should be described in the vendor's guid- 
ance material to ensure that the user can accurately interpret the displayed data. 

3. To ensure airman compliance with Federal Aviation Regulations, National Air- 
space System (NAS) status products (such as NOTAM's, Special Use Airspace Status, 
etc.) and other government flight information should include verbatim transmissions of 
FAA products. If these products are modified, the modification process, and any limita- 
tions of the resultant product should be described in the vendor's user guidance. 
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